- Open account settings.
- Scroll down to API and click Create New API Token.
- Open up
kaggle.jsonand use the username and key.
This notebook is a template with each step that you need to complete for the project.
Please fill in your code where there are explicit ? markers in the notebook. You are welcome to add more cells and code as you see fit.
Once you have completed all the code implementations, please export your notebook as a HTML file so the reviews can view your code. Make sure you have all outputs correctly outputted.
File-> Export Notebook As... -> Export Notebook as HTML
There is a writeup to complete as well after all code implememtation is done. Please answer all questions and attach the necessary tables and charts. You can complete the writeup in either markdown or PDF.
Completing the code template and writeup template will cover all of the rubric points for this project.
The rubric contains "Stand Out Suggestions" for enhancing the project beyond the minimum requirements. The stand out suggestions are optional. If you decide to pursue the "stand out suggestions", you can include the code in this notebook and also discuss the results in the writeup file.
Below is example of steps to get the API username and key. Each student will have their own username and key.
kaggle.json and use the username and key.
ml.t3.medium instance (2 vCPU + 4 GiB)Python 3 (MXNet 1.8 Python 3.7 CPU Optimized)!pip install -U pip
!pip install -U setuptools wheel
!pip install -U "mxnet<2.0.0" bokeh==2.0.1
!pip install autogluon --no-cache-dir
# Without --no-cache-dir, smaller aws instances may have trouble installing
Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (21.3.1)
Collecting pip
Using cached pip-23.1.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 21.3.1
Uninstalling pip-21.3.1:
Successfully uninstalled pip-21.3.1
Successfully installed pip-23.1.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (59.3.0)
Collecting setuptools
Using cached setuptools-67.8.0-py3-none-any.whl (1.1 MB)
Collecting wheel
Using cached wheel-0.40.0-py3-none-any.whl (64 kB)
Installing collected packages: wheel, setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 59.3.0
Uninstalling setuptools-59.3.0:
Successfully uninstalled setuptools-59.3.0
Successfully installed setuptools-59.4.0 wheel-0.40.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Collecting mxnet<2.0.0
Using cached mxnet-1.9.1-py3-none-manylinux2014_x86_64.whl (49.1 MB)
Collecting bokeh==2.0.1
Using cached bokeh-2.0.1-py3-none-any.whl
Requirement already satisfied: PyYAML>=3.10 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (5.4.1)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (2.8.0)
Requirement already satisfied: Jinja2>=2.7 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (3.0.3)
Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (1.19.1)
Requirement already satisfied: pillow>=4.0 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (8.4.0)
Requirement already satisfied: packaging>=16.8 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (21.3)
Requirement already satisfied: tornado>=5 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (6.0.4)
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (4.0.1)
Requirement already satisfied: requests<3,>=2.20.0 in /usr/local/lib/python3.7/site-packages (from mxnet<2.0.0) (2.22.0)
Requirement already satisfied: graphviz<0.9.0,>=0.8.1 in /usr/local/lib/python3.7/site-packages (from mxnet<2.0.0) (0.8.4)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/site-packages (from Jinja2>=2.7->bokeh==2.0.1) (2.0.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/site-packages (from packaging>=16.8->bokeh==2.0.1) (3.0.6)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/site-packages (from python-dateutil>=2.1->bokeh==2.0.1) (1.16.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (1.25.11)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (2021.10.8)
Installing collected packages: mxnet, bokeh
Attempting uninstall: bokeh
Found existing installation: bokeh 2.4.2
Uninstalling bokeh-2.4.2:
Successfully uninstalled bokeh-2.4.2
Successfully installed bokeh-2.0.1 mxnet-1.9.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Collecting autogluon
Downloading autogluon-0.6.2-py3-none-any.whl (9.8 kB)
Collecting autogluon.core[all]==0.6.2 (from autogluon)
Downloading autogluon.core-0.6.2-py3-none-any.whl (226 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 226.5/226.5 kB 16.8 MB/s eta 0:00:00
Collecting autogluon.features==0.6.2 (from autogluon)
Downloading autogluon.features-0.6.2-py3-none-any.whl (60 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.0/60.0 kB 174.5 MB/s eta 0:00:00
Collecting autogluon.tabular[all]==0.6.2 (from autogluon)
Downloading autogluon.tabular-0.6.2-py3-none-any.whl (292 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 292.5/292.5 kB 221.4 MB/s eta 0:00:00
Collecting autogluon.multimodal==0.6.2 (from autogluon)
Downloading autogluon.multimodal-0.6.2-py3-none-any.whl (303 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 303.4/303.4 kB 232.3 MB/s eta 0:00:00
Collecting autogluon.text==0.6.2 (from autogluon)
Downloading autogluon.text-0.6.2-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.1/62.1 kB 171.4 MB/s eta 0:00:00
Collecting autogluon.vision==0.6.2 (from autogluon)
Downloading autogluon.vision-0.6.2-py3-none-any.whl (49 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.8/49.8 kB 160.5 MB/s eta 0:00:00
Collecting autogluon.timeseries[all]==0.6.2 (from autogluon)
Downloading autogluon.timeseries-0.6.2-py3-none-any.whl (103 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 103.6/103.6 kB 192.8 MB/s eta 0:00:00
Collecting numpy<1.24,>=1.21 (from autogluon.core[all]==0.6.2->autogluon)
Downloading numpy-1.21.6-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.7/15.7 MB 172.8 MB/s eta 0:00:00a 0:00:01
Collecting scipy<1.10.0,>=1.5.4 (from autogluon.core[all]==0.6.2->autogluon)
Downloading scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 38.1/38.1 MB 156.3 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: scikit-learn<1.2,>=1.0.0 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (1.0.1)
Requirement already satisfied: psutil<6,>=5.7.3 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (5.8.0)
Requirement already satisfied: networkx<3.0,>=2.3 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (2.6.3)
Requirement already satisfied: pandas!=1.4.0,<1.6,>=1.2.5 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (1.3.4)
Requirement already satisfied: tqdm>=4.38.0 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (4.39.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (2.22.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (3.5.0)
Collecting dask<=2021.11.2,>=2021.09.1 (from autogluon.core[all]==0.6.2->autogluon)
Downloading dask-2021.11.2-py3-none-any.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 211.4 MB/s eta 0:00:00
Collecting distributed<=2021.11.2,>=2021.09.1 (from autogluon.core[all]==0.6.2->autogluon)
Downloading distributed-2021.11.2-py3-none-any.whl (802 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 802.2/802.2 kB 215.3 MB/s eta 0:00:00
Requirement already satisfied: boto3 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.6.2->autogluon) (1.20.17)
Collecting autogluon.common==0.6.2 (from autogluon.core[all]==0.6.2->autogluon)
Downloading autogluon.common-0.6.2-py3-none-any.whl (44 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.7/44.7 kB 148.7 MB/s eta 0:00:00
Collecting hyperopt<0.2.8,>=0.2.7 (from autogluon.core[all]==0.6.2->autogluon)
Downloading hyperopt-0.2.7-py2.py3-none-any.whl (1.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 201.1 MB/s eta 0:00:00
Collecting ray[tune]<2.1,>=2.0 (from autogluon.core[all]==0.6.2->autogluon)
Downloading ray-2.0.1-cp37-cp37m-manylinux2014_x86_64.whl (60.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.5/60.5 MB 178.8 MB/s eta 0:00:00a 0:00:01
Collecting Pillow<=9.4.0,>=9.3.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading Pillow-9.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 182.8 MB/s eta 0:00:00
Collecting jsonschema<=4.8.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading jsonschema-4.8.0-py3-none-any.whl (81 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.4/81.4 kB 181.0 MB/s eta 0:00:00
Collecting seqeval<=1.2.2 (from autogluon.multimodal==0.6.2->autogluon)
Downloading seqeval-1.2.2.tar.gz (43 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.6/43.6 kB 116.2 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting evaluate<=0.3.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading evaluate-0.3.0-py3-none-any.whl (72 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.9/72.9 kB 145.1 MB/s eta 0:00:00
Collecting accelerate<0.14,>=0.9 (from autogluon.multimodal==0.6.2->autogluon)
Downloading accelerate-0.13.2-py3-none-any.whl (148 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 148.8/148.8 kB 193.9 MB/s eta 0:00:00
Collecting timm<0.7.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading timm-0.6.13-py3-none-any.whl (549 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 549.1/549.1 kB 223.1 MB/s eta 0:00:00
Collecting torch<1.13,>=1.9 (from autogluon.multimodal==0.6.2->autogluon)
Downloading torch-1.12.1-cp37-cp37m-manylinux1_x86_64.whl (776.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 776.3/776.3 MB 145.0 MB/s eta 0:00:0000:0100:01
Collecting torchvision<0.14.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading torchvision-0.13.1-cp37-cp37m-manylinux1_x86_64.whl (19.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.1/19.1 MB 179.2 MB/s eta 0:00:0000:0100:01
Collecting torchtext<0.14.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading torchtext-0.13.1-cp37-cp37m-manylinux1_x86_64.whl (1.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 179.4 MB/s eta 0:00:00
Collecting fairscale<=0.4.6,>=0.4.5 (from autogluon.multimodal==0.6.2->autogluon)
Downloading fairscale-0.4.6.tar.gz (248 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 248.2/248.2 kB 211.8 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting scikit-image<0.20.0,>=0.19.1 (from autogluon.multimodal==0.6.2->autogluon)
Downloading scikit_image-0.19.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.5/13.5 MB 156.4 MB/s eta 0:00:00a 0:00:01
Collecting smart-open<5.3.0,>=5.2.1 (from autogluon.multimodal==0.6.2->autogluon)
Downloading smart_open-5.2.1-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.6/58.6 kB 101.6 MB/s eta 0:00:00
Collecting pytorch-lightning<1.8.0,>=1.7.4 (from autogluon.multimodal==0.6.2->autogluon)
Downloading pytorch_lightning-1.7.7-py3-none-any.whl (708 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 708.1/708.1 kB 222.5 MB/s eta 0:00:00
Collecting text-unidecode<=1.3 (from autogluon.multimodal==0.6.2->autogluon)
Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.2/78.2 kB 182.8 MB/s eta 0:00:00
Collecting torchmetrics<0.9.0,>=0.8.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading torchmetrics-0.8.2-py3-none-any.whl (409 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 409.8/409.8 kB 228.3 MB/s eta 0:00:00
Collecting transformers<4.24.0,>=4.23.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading transformers-4.23.1-py3-none-any.whl (5.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 217.2 MB/s eta 0:00:00
Collecting nptyping<1.5.0,>=1.4.4 (from autogluon.multimodal==0.6.2->autogluon)
Downloading nptyping-1.4.4-py3-none-any.whl (31 kB)
Collecting omegaconf<2.2.0,>=2.1.1 (from autogluon.multimodal==0.6.2->autogluon)
Downloading omegaconf-2.1.2-py3-none-any.whl (74 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 74.7/74.7 kB 180.5 MB/s eta 0:00:00
Collecting sentencepiece<0.2.0,>=0.1.95 (from autogluon.multimodal==0.6.2->autogluon)
Downloading sentencepiece-0.1.99-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 183.8 MB/s eta 0:00:00
Collecting pytorch-metric-learning<1.4.0,>=1.3.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading pytorch_metric_learning-1.3.2-py3-none-any.whl (109 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 109.4/109.4 kB 200.2 MB/s eta 0:00:00
Collecting nlpaug<=1.1.10,>=1.1.10 (from autogluon.multimodal==0.6.2->autogluon)
Downloading nlpaug-1.1.10-py3-none-any.whl (410 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 410.8/410.8 kB 224.6 MB/s eta 0:00:00
Collecting nltk<4.0.0,>=3.4.5 (from autogluon.multimodal==0.6.2->autogluon)
Downloading nltk-3.8.1-py3-none-any.whl (1.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 245.1 MB/s eta 0:00:00
Collecting openmim<=0.2.1,>0.1.5 (from autogluon.multimodal==0.6.2->autogluon)
Downloading openmim-0.2.1-py2.py3-none-any.whl (49 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.7/49.7 kB 88.4 MB/s eta 0:00:00
Collecting defusedxml<=0.7.1,>=0.7.1 (from autogluon.multimodal==0.6.2->autogluon)
Downloading defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting albumentations<=1.2.0,>=1.1.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading albumentations-1.2.0-py3-none-any.whl (113 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 113.5/113.5 kB 181.6 MB/s eta 0:00:00
Collecting catboost<1.2,>=1.0 (from autogluon.tabular[all]==0.6.2->autogluon)
Downloading catboost-1.1.1-cp37-none-manylinux1_x86_64.whl (76.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76.6/76.6 MB 179.3 MB/s eta 0:00:0000:0100:01
Collecting xgboost<1.8,>=1.6 (from autogluon.tabular[all]==0.6.2->autogluon)
Downloading xgboost-1.6.2-py3-none-manylinux2014_x86_64.whl (255.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 255.9/255.9 MB 175.7 MB/s eta 0:00:0000:0100:01
Collecting lightgbm<3.4,>=3.3 (from autogluon.tabular[all]==0.6.2->autogluon)
Downloading lightgbm-3.3.5-py3-none-manylinux1_x86_64.whl (2.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 131.0 MB/s eta 0:00:00
Collecting fastai<2.8,>=2.3.1 (from autogluon.tabular[all]==0.6.2->autogluon)
Downloading fastai-2.7.12-py3-none-any.whl (233 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 233.1/233.1 kB 220.2 MB/s eta 0:00:00
Requirement already satisfied: joblib~=1.1 in /usr/local/lib/python3.7/site-packages (from autogluon.timeseries[all]==0.6.2->autogluon) (1.1.0)
Collecting statsmodels~=0.13.0 (from autogluon.timeseries[all]==0.6.2->autogluon)
Downloading statsmodels-0.13.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.9/9.9 MB 172.3 MB/s eta 0:00:00a 0:00:01
Collecting gluonts~=0.11.0 (from autogluon.timeseries[all]==0.6.2->autogluon)
Downloading gluonts-0.11.12-py3-none-any.whl (1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 234.8 MB/s eta 0:00:00
Collecting pmdarima~=1.8.2 (from autogluon.timeseries[all]==0.6.2->autogluon)
Downloading pmdarima-1.8.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (1.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 240.6 MB/s eta 0:00:00
Collecting sktime<0.14,>=0.13.1 (from autogluon.timeseries[all]==0.6.2->autogluon)
Downloading sktime-0.13.4-py3-none-any.whl (7.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.0/7.0 MB 148.9 MB/s eta 0:00:00a 0:00:01
Collecting tbats~=1.1 (from autogluon.timeseries[all]==0.6.2->autogluon)
Downloading tbats-1.1.3-py3-none-any.whl (44 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.0/44.0 kB 143.5 MB/s eta 0:00:00
Collecting gluoncv<0.10.6,>=0.10.5 (from autogluon.vision==0.6.2->autogluon)
Downloading gluoncv-0.10.5.post0-py2.py3-none-any.whl (1.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 218.1 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from autogluon.common==0.6.2->autogluon.core[all]==0.6.2->autogluon) (59.4.0)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/site-packages (from accelerate<0.14,>=0.9->autogluon.multimodal==0.6.2->autogluon) (21.3)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/site-packages (from accelerate<0.14,>=0.9->autogluon.multimodal==0.6.2->autogluon) (5.4.1)
INFO: pip is looking at multiple versions of albumentations to determine which version is compatible with other requirements. This could take a while.
Collecting albumentations<=1.2.0,>=1.1.0 (from autogluon.multimodal==0.6.2->autogluon)
Downloading albumentations-1.1.0-py3-none-any.whl (102 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.4/102.4 kB 192.9 MB/s eta 0:00:00
Collecting qudida>=0.0.4 (from albumentations<=1.2.0,>=1.1.0->autogluon.multimodal==0.6.2->autogluon)
Downloading qudida-0.0.4-py3-none-any.whl (3.5 kB)
Collecting opencv-python-headless>=4.1.1 (from albumentations<=1.2.0,>=1.1.0->autogluon.multimodal==0.6.2->autogluon)
Downloading opencv_python_headless-4.7.0.72-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (49.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.2/49.2 MB 176.6 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: graphviz in /usr/local/lib/python3.7/site-packages (from catboost<1.2,>=1.0->autogluon.tabular[all]==0.6.2->autogluon) (0.8.4)
Requirement already satisfied: plotly in /usr/local/lib/python3.7/site-packages (from catboost<1.2,>=1.0->autogluon.tabular[all]==0.6.2->autogluon) (5.4.0)
Requirement already satisfied: six in /usr/local/lib/python3.7/site-packages (from catboost<1.2,>=1.0->autogluon.tabular[all]==0.6.2->autogluon) (1.16.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon) (2.0.0)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon) (2021.11.1)
Collecting partd>=0.3.10 (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading partd-1.4.0-py3-none-any.whl (18 kB)
Collecting toolz>=0.8.2 (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading toolz-0.12.0-py3-none-any.whl (55 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.8/55.8 kB 153.0 MB/s eta 0:00:00
Collecting click>=6.6 (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading click-8.1.3-py3-none-any.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 172.0 MB/s eta 0:00:00
Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon) (3.0.3)
Collecting msgpack>=0.6.0 (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading msgpack-1.0.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (299 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 299.7/299.7 kB 232.3 MB/s eta 0:00:00
Collecting sortedcontainers!=2.0.0,!=2.0.1 (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting tblib>=1.6.0 (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading tblib-1.7.0-py2.py3-none-any.whl (12 kB)
Collecting zict>=0.1.3 (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading zict-2.2.0-py2.py3-none-any.whl (23 kB)
Requirement already satisfied: tornado>=5 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon) (6.0.4)
Collecting datasets>=2.0.0 (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading datasets-2.12.0-py3-none-any.whl (474 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 474.6/474.6 kB 238.1 MB/s eta 0:00:00
Requirement already satisfied: dill in /usr/local/lib/python3.7/site-packages (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon) (0.3.4)
Collecting tqdm>=4.38.0 (from autogluon.core[all]==0.6.2->autogluon)
Downloading tqdm-4.65.0-py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.1/77.1 kB 180.7 MB/s eta 0:00:00
Collecting xxhash (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading xxhash-3.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (213 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 213.1/213.1 kB 212.8 MB/s eta 0:00:00
Requirement already satisfied: multiprocess in /usr/local/lib/python3.7/site-packages (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon) (0.70.12.2)
Collecting huggingface-hub>=0.7.0 (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading huggingface_hub-0.15.1-py3-none-any.whl (236 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 236.8/236.8 kB 224.3 MB/s eta 0:00:00
Collecting responses<0.19 (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/site-packages (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon) (4.8.2)
Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon) (23.1.2)
Collecting fastdownload<2,>=0.0.5 (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading fastdownload-0.0.7-py3-none-any.whl (12 kB)
Collecting fastcore<1.6,>=1.5.29 (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading fastcore-1.5.29-py3-none-any.whl (67 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.6/67.6 kB 172.3 MB/s eta 0:00:00
Collecting fastprogress>=0.2.4 (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading fastprogress-1.0.3-py3-none-any.whl (12 kB)
Collecting spacy<4 (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading spacy-3.5.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.5 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 216.2 MB/s eta 0:00:00
Requirement already satisfied: portalocker in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.6.2->autogluon) (2.3.2)
Collecting yacs (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.6.2->autogluon)
Downloading yacs-0.1.8-py3-none-any.whl (14 kB)
Collecting autocfg (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.6.2->autogluon)
Downloading autocfg-0.0.8-py3-none-any.whl (13 kB)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.6.2->autogluon) (4.5.4.60)
Collecting pydantic~=1.7 (from gluonts~=0.11.0->autogluon.timeseries[all]==0.6.2->autogluon)
Downloading pydantic-1.10.9-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 225.6 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.7/site-packages (from gluonts~=0.11.0->autogluon.timeseries[all]==0.6.2->autogluon) (4.0.1)
Collecting future (from hyperopt<0.2.8,>=0.2.7->autogluon.core[all]==0.6.2->autogluon)
Downloading future-0.18.3.tar.gz (840 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 840.9/840.9 kB 238.5 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting py4j (from hyperopt<0.2.8,>=0.2.7->autogluon.core[all]==0.6.2->autogluon)
Downloading py4j-0.10.9.7-py2.py3-none-any.whl (200 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 200.5/200.5 kB 200.7 MB/s eta 0:00:00
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.7/site-packages (from jsonschema<=4.8.0->autogluon.multimodal==0.6.2->autogluon) (21.2.0)
Collecting importlib-resources>=1.4.0 (from jsonschema<=4.8.0->autogluon.multimodal==0.6.2->autogluon)
Downloading importlib_resources-5.12.0-py3-none-any.whl (36 kB)
Collecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 (from jsonschema<=4.8.0->autogluon.multimodal==0.6.2->autogluon)
Downloading pyrsistent-0.19.3-py3-none-any.whl (57 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.5/57.5 kB 162.6 MB/s eta 0:00:00
Requirement already satisfied: wheel in /usr/local/lib/python3.7/site-packages (from lightgbm<3.4,>=3.3->autogluon.tabular[all]==0.6.2->autogluon) (0.40.0)
Collecting regex>=2021.8.3 (from nltk<4.0.0,>=3.4.5->autogluon.multimodal==0.6.2->autogluon)
Downloading regex-2023.6.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (755 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 755.7/755.7 kB 238.4 MB/s eta 0:00:00
Collecting typish>=1.7.0 (from nptyping<1.5.0,>=1.4.4->autogluon.multimodal==0.6.2->autogluon)
Downloading typish-1.9.3-py3-none-any.whl (45 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.1/45.1 kB 120.4 MB/s eta 0:00:00
Collecting antlr4-python3-runtime==4.8 (from omegaconf<2.2.0,>=2.1.1->autogluon.multimodal==0.6.2->autogluon)
Downloading antlr4-python3-runtime-4.8.tar.gz (112 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 112.4/112.4 kB 131.6 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: colorama in /usr/local/lib/python3.7/site-packages (from openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon) (0.4.3)
Collecting model-index (from openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon)
Downloading model_index-0.1.11-py3-none-any.whl (34 kB)
Collecting rich (from openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon)
Downloading rich-13.4.1-py3-none-any.whl (239 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 239.4/239.4 kB 143.6 MB/s eta 0:00:00
Requirement already satisfied: tabulate in /usr/local/lib/python3.7/site-packages (from openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon) (0.8.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/site-packages (from pandas!=1.4.0,<1.6,>=1.2.5->autogluon.core[all]==0.6.2->autogluon) (2.8.0)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/site-packages (from pandas!=1.4.0,<1.6,>=1.2.5->autogluon.core[all]==0.6.2->autogluon) (2021.3)
Requirement already satisfied: Cython!=0.29.18,>=0.29 in /usr/local/lib/python3.7/site-packages (from pmdarima~=1.8.2->autogluon.timeseries[all]==0.6.2->autogluon) (0.29.24)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/site-packages (from pmdarima~=1.8.2->autogluon.timeseries[all]==0.6.2->autogluon) (1.25.11)
Collecting tensorboard>=2.9.1 (from pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading tensorboard-2.11.2-py3-none-any.whl (6.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.0/6.0 MB 212.9 MB/s eta 0:00:00
Collecting pyDeprecate>=0.3.1 (from pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading pyDeprecate-0.3.2-py3-none-any.whl (10 kB)
Collecting click>=6.6 (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading click-8.0.4-py3-none-any.whl (97 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.5/97.5 kB 193.8 MB/s eta 0:00:00
Collecting filelock (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading filelock-3.12.1-py3-none-any.whl (10 kB)
Requirement already satisfied: protobuf<4.0.0,>=3.15.3 in /usr/local/lib/python3.7/site-packages (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon) (3.19.1)
Collecting aiosignal (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting frozenlist (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading frozenlist-1.3.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (148 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 148.0/148.0 kB 207.9 MB/s eta 0:00:00
Collecting virtualenv (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading virtualenv-20.23.0-py3-none-any.whl (3.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 228.3 MB/s eta 0:00:00
Collecting grpcio<=1.43.0,>=1.32.0 (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading grpcio-1.43.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.1/4.1 MB 206.0 MB/s eta 0:00:00
Collecting tensorboardX>=1.9 (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading tensorboardX-2.6-py2.py3-none-any.whl (114 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.5/114.5 kB 189.0 MB/s eta 0:00:00
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.6.2->autogluon) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.6.2->autogluon) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.6.2->autogluon) (2021.10.8)
Requirement already satisfied: imageio>=2.4.1 in /usr/local/lib/python3.7/site-packages (from scikit-image<0.20.0,>=0.19.1->autogluon.multimodal==0.6.2->autogluon) (2.13.1)
Collecting tifffile>=2019.7.26 (from scikit-image<0.20.0,>=0.19.1->autogluon.multimodal==0.6.2->autogluon)
Downloading tifffile-2021.11.2-py3-none-any.whl (178 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 178.9/178.9 kB 214.7 MB/s eta 0:00:00
Collecting PyWavelets>=1.1.1 (from scikit-image<0.20.0,>=0.19.1->autogluon.multimodal==0.6.2->autogluon)
Downloading PyWavelets-1.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.4/6.4 MB 193.7 MB/s eta 0:00:00
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/site-packages (from scikit-learn<1.2,>=1.0.0->autogluon.core[all]==0.6.2->autogluon) (3.0.0)
Collecting deprecated>=1.2.13 (from sktime<0.14,>=0.13.1->autogluon.timeseries[all]==0.6.2->autogluon)
Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: numba>=0.53 in /usr/local/lib/python3.7/site-packages (from sktime<0.14,>=0.13.1->autogluon.timeseries[all]==0.6.2->autogluon) (0.53.1)
Collecting patsy>=0.5.2 (from statsmodels~=0.13.0->autogluon.timeseries[all]==0.6.2->autogluon)
Downloading patsy-0.5.3-py2.py3-none-any.whl (233 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 233.8/233.8 kB 223.4 MB/s eta 0:00:00
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers<4.24.0,>=4.23.0->autogluon.multimodal==0.6.2->autogluon)
Downloading tokenizers-0.13.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 192.5 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: botocore<1.24.0,>=1.23.17 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.6.2->autogluon) (1.23.17)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.6.2->autogluon) (0.10.0)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.6.2->autogluon) (0.5.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.6.2->autogluon) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.6.2->autogluon) (4.28.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.6.2->autogluon) (1.3.2)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.6.2->autogluon) (3.0.6)
Requirement already satisfied: setuptools-scm>=4 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.6.2->autogluon) (6.3.2)
Collecting pyarrow>=8.0.0 (from datasets>=2.0.0->evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading pyarrow-12.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (39.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.1/39.1 MB 195.9 MB/s eta 0:00:00a 0:00:01
Collecting aiohttp (from datasets>=2.0.0->evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading aiohttp-3.8.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (948 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 948.1/948.1 kB 242.9 MB/s eta 0:00:00
Collecting wrapt<2,>=1.10 (from deprecated>=1.2.13->sktime<0.14,>=0.13.1->autogluon.timeseries[all]==0.6.2->autogluon)
Downloading wrapt-1.15.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.7/75.7 kB 180.5 MB/s eta 0:00:00
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/site-packages (from importlib-resources>=1.4.0->jsonschema<=4.8.0->autogluon.multimodal==0.6.2->autogluon) (3.6.0)
Requirement already satisfied: llvmlite<0.37,>=0.36.0rc1 in /usr/local/lib/python3.7/site-packages (from numba>=0.53->sktime<0.14,>=0.13.1->autogluon.timeseries[all]==0.6.2->autogluon) (0.36.0)
Collecting locket (from partd>=0.3.10->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading locket-1.0.0-py2.py3-none-any.whl (4.4 kB)
Collecting typing-extensions~=4.0 (from gluonts~=0.11.0->autogluon.timeseries[all]==0.6.2->autogluon)
Downloading typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Requirement already satisfied: tomli>=1.0.0 in /usr/local/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib->autogluon.core[all]==0.6.2->autogluon) (1.2.2)
Collecting spacy-legacy<3.1.0,>=3.0.11 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading spacy_legacy-3.0.12-py2.py3-none-any.whl (29 kB)
Collecting spacy-loggers<2.0.0,>=1.0.0 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading spacy_loggers-1.0.4-py3-none-any.whl (11 kB)
Collecting murmurhash<1.1.0,>=0.28.0 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading murmurhash-1.0.9-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (21 kB)
Collecting cymem<2.1.0,>=2.0.2 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading cymem-2.0.7-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (36 kB)
Collecting preshed<3.1.0,>=3.0.2 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading preshed-3.0.8-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (126 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.6/126.6 kB 190.7 MB/s eta 0:00:00
Collecting thinc<8.2.0,>=8.1.8 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading thinc-8.1.10-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (914 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 914.3/914.3 kB 237.2 MB/s eta 0:00:00
Collecting wasabi<1.2.0,>=0.9.1 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading wasabi-1.1.2-py3-none-any.whl (27 kB)
Collecting srsly<3.0.0,>=2.4.3 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading srsly-2.4.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (490 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 490.9/490.9 kB 231.1 MB/s eta 0:00:00
Collecting catalogue<2.1.0,>=2.0.6 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading catalogue-2.0.8-py3-none-any.whl (17 kB)
Collecting typer<0.8.0,>=0.3.0 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading typer-0.7.0-py3-none-any.whl (38 kB)
Collecting pathy>=0.10.0 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading pathy-0.10.1-py3-none-any.whl (48 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 160.0 MB/s eta 0:00:00
Collecting langcodes<4.0.0,>=3.2.0 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading langcodes-3.3.0-py3-none-any.whl (181 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.6/181.6 kB 217.7 MB/s eta 0:00:00
Collecting typing-extensions~=4.0 (from gluonts~=0.11.0->autogluon.timeseries[all]==0.6.2->autogluon)
Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting absl-py>=0.4 (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.5/126.5 kB 181.9 MB/s eta 0:00:00
Collecting google-auth<3,>=1.6.3 (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading google_auth-2.19.1-py2.py3-none-any.whl (181 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 206.0 MB/s eta 0:00:00
Collecting google-auth-oauthlib<0.5,>=0.4.1 (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8 (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading Markdown-3.4.3-py3-none-any.whl (93 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.9/93.9 kB 187.9 MB/s eta 0:00:00
Collecting tensorboard-data-server<0.7.0,>=0.6.0 (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.9/4.9 MB 221.4 MB/s eta 0:00:00
Collecting tensorboard-plugin-wit>=1.6.0 (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 781.3/781.3 kB 223.1 MB/s eta 0:00:00
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon) (2.0.2)
Collecting heapdict (from zict>=0.1.3->distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon)
Downloading HeapDict-1.0.1-py3-none-any.whl (3.9 kB)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/site-packages (from jinja2->distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.6.2->autogluon) (2.0.1)
Collecting ordered-set (from model-index->openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon)
Downloading ordered_set-4.1.0-py3-none-any.whl (7.6 kB)
Requirement already satisfied: tenacity>=6.2.0 in /usr/local/lib/python3.7/site-packages (from plotly->catboost<1.2,>=1.0->autogluon.tabular[all]==0.6.2->autogluon) (8.0.1)
Collecting markdown-it-py<3.0.0,>=2.2.0 (from rich->openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon)
Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.5/84.5 kB 184.3 MB/s eta 0:00:00
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/site-packages (from rich->openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon) (2.14.0)
Collecting distlib<1,>=0.3.6 (from virtualenv->ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading distlib-0.3.6-py2.py3-none-any.whl (468 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.5/468.5 kB 240.5 MB/s eta 0:00:00
Collecting importlib-metadata (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading importlib_metadata-6.6.0-py3-none-any.whl (22 kB)
Collecting platformdirs<4,>=3.2 (from virtualenv->ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading platformdirs-3.5.3-py3-none-any.whl (15 kB)
Collecting cachetools<6.0,>=2.0.0 (from google-auth<3,>=1.6.3->tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth<3,>=1.6.3->tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.3/181.3 kB 208.0 MB/s eta 0:00:00
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon) (4.7.2)
Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting mdurl~=0.1 (from markdown-it-py<3.0.0,>=2.2.0->rich->openmim<=0.2.1,>0.1.5->autogluon.multimodal==0.6.2->autogluon)
Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
INFO: pip is looking at multiple versions of platformdirs to determine which version is compatible with other requirements. This could take a while.
Collecting platformdirs<4,>=3.2 (from virtualenv->ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading platformdirs-3.5.2-py3-none-any.whl (15 kB)
Downloading platformdirs-3.5.1-py3-none-any.whl (15 kB)
Downloading platformdirs-3.5.0-py3-none-any.whl (15 kB)
Downloading platformdirs-3.4.0-py3-none-any.whl (15 kB)
Downloading platformdirs-3.3.0-py3-none-any.whl (15 kB)
Downloading platformdirs-3.2.0-py3-none-any.whl (14 kB)
Collecting markdown>=2.6.8 (from tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading Markdown-3.4.2-py3-none-any.whl (93 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.9/93.9 kB 187.4 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of platformdirs to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Collecting catalogue<2.1.0,>=2.0.6 (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading catalogue-2.0.7-py3-none-any.whl (17 kB)
Downloading catalogue-2.0.6-py3-none-any.whl (17 kB)
Collecting importlib-metadata (from evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading importlib_metadata-6.5.1-py3-none-any.whl (22 kB)
Downloading importlib_metadata-6.5.0-py3-none-any.whl (22 kB)
Downloading importlib_metadata-6.4.1-py3-none-any.whl (22 kB)
Collecting virtualenv (from ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading virtualenv-20.22.0-py3-none-any.whl (3.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 141.7 MB/s eta 0:00:00
Downloading virtualenv-20.21.1-py3-none-any.whl (8.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.7/8.7 MB 185.9 MB/s eta 0:00:00a 0:00:01
Collecting platformdirs<4,>=2.4 (from virtualenv->ray[tune]<2.1,>=2.0->autogluon.core[all]==0.6.2->autogluon)
Downloading platformdirs-3.1.1-py3-none-any.whl (14 kB)
Collecting blis<0.8.0,>=0.7.8 (from thinc<8.2.0,>=8.1.8->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading blis-0.7.9-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.2/10.2 MB 188.3 MB/s eta 0:00:00a 0:00:01
Collecting confection<1.0.0,>=0.0.1 (from thinc<8.2.0,>=8.1.8->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==0.6.2->autogluon)
Downloading confection-0.0.4-py3-none-any.whl (32 kB)
Collecting charset-normalizer<4.0,>=2.0 (from aiohttp->datasets>=2.0.0->evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading charset_normalizer-3.1.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (171 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 171.0/171.0 kB 138.1 MB/s eta 0:00:00
Collecting multidict<7.0,>=4.5 (from aiohttp->datasets>=2.0.0->evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading multidict-6.0.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (94 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.8/94.8 kB 167.9 MB/s eta 0:00:00
Collecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->datasets>=2.0.0->evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting yarl<2.0,>=1.0 (from aiohttp->datasets>=2.0.0->evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading yarl-1.9.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (236 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 236.2/236.2 kB 134.6 MB/s eta 0:00:00
Collecting asynctest==0.13.0 (from aiohttp->datasets>=2.0.0->evaluate<=0.3.0->autogluon.multimodal==0.6.2->autogluon)
Downloading asynctest-0.13.0-py3-none-any.whl (26 kB)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon) (0.4.8)
Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.9.1->pytorch-lightning<1.8.0,>=1.7.4->autogluon.multimodal==0.6.2->autogluon)
Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 kB 211.9 MB/s eta 0:00:00
Building wheels for collected packages: fairscale, antlr4-python3-runtime, seqeval, future
Building wheel for fairscale (pyproject.toml) ... done
Created wheel for fairscale: filename=fairscale-0.4.6-py3-none-any.whl size=307224 sha256=84a9601561d344c4ada110c45c487e383263d0f773ae684ad00e33ece1e44320
Stored in directory: /tmp/pip-ephem-wheel-cache-91bpx3zq/wheels/4e/4f/0b/94c29ea06dfad93260cb0377855f87b7b863312317a7f69fe7
Building wheel for antlr4-python3-runtime (setup.py) ... done
Created wheel for antlr4-python3-runtime: filename=antlr4_python3_runtime-4.8-py3-none-any.whl size=141210 sha256=38a42af6d16addf45e9ca57f1f01fd06d4f067a5d44424402b28baf8364602f9
Stored in directory: /tmp/pip-ephem-wheel-cache-91bpx3zq/wheels/ca/33/b7/336836125fc9bb4ceaa4376d8abca10ca8bc84ddc824baea6c
Building wheel for seqeval (setup.py) ... done
Created wheel for seqeval: filename=seqeval-1.2.2-py3-none-any.whl size=16165 sha256=98c1618a23c05264823a5563669b9fb99574c532c6d638ea18e4e58481e3a0ff
Stored in directory: /tmp/pip-ephem-wheel-cache-91bpx3zq/wheels/05/96/ee/7cac4e74f3b19e3158dce26a20a1c86b3533c43ec72a549fd7
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492022 sha256=22548e57f1ec838c8cee917068d2c8851446a5ece768c00c20677af8658873c3
Stored in directory: /tmp/pip-ephem-wheel-cache-91bpx3zq/wheels/fa/cd/1f/c6b7b50b564983bf3011e8fc75d06047ddc50c07f6e3660b00
Successfully built fairscale antlr4-python3-runtime seqeval future
Installing collected packages: typish, tokenizers, text-unidecode, tensorboard-plugin-wit, sortedcontainers, sentencepiece, py4j, msgpack, heapdict, distlib, cymem, antlr4-python3-runtime, zict, yacs, xxhash, wrapt, typing-extensions, tqdm, toolz, tensorboard-data-server, tblib, spacy-loggers, spacy-legacy, smart-open, regex, pyrsistent, pyDeprecate, pyasn1-modules, Pillow, ordered-set, omegaconf, oauthlib, numpy, murmurhash, multidict, mdurl, locket, langcodes, importlib-resources, grpcio, future, frozenlist, filelock, fastprogress, defusedxml, charset-normalizer, cachetools, autocfg, asynctest, absl-py, yarl, wasabi, torch, tifffile, tensorboardX, scipy, responses, requests-oauthlib, PyWavelets, pydantic, pyarrow, preshed, platformdirs, patsy, partd, opencv-python-headless, nptyping, markdown-it-py, importlib-metadata, google-auth, fastcore, deprecated, catalogue, blis, async-timeout, aiosignal, xgboost, virtualenv, torchvision, torchtext, torchmetrics, statsmodels, srsly, scikit-image, rich, nlpaug, markdown, jsonschema, hyperopt, huggingface-hub, google-auth-oauthlib, gluonts, fastdownload, fairscale, dask, click, aiohttp, accelerate, typer, transformers, timm, tensorboard, sktime, seqeval, ray, qudida, pytorch-metric-learning, pmdarima, nltk, model-index, lightgbm, gluoncv, distributed, confection, catboost, thinc, tbats, pytorch-lightning, pathy, openmim, datasets, autogluon.common, albumentations, spacy, evaluate, autogluon.features, autogluon.core, fastai, autogluon.tabular, autogluon.multimodal, autogluon.vision, autogluon.timeseries, autogluon.text, autogluon
Attempting uninstall: typing-extensions
Found existing installation: typing_extensions 4.0.1
Uninstalling typing_extensions-4.0.1:
Successfully uninstalled typing_extensions-4.0.1
Attempting uninstall: tqdm
Found existing installation: tqdm 4.39.0
Uninstalling tqdm-4.39.0:
Successfully uninstalled tqdm-4.39.0
Attempting uninstall: Pillow
Found existing installation: Pillow 8.4.0
Uninstalling Pillow-8.4.0:
Successfully uninstalled Pillow-8.4.0
Attempting uninstall: numpy
Found existing installation: numpy 1.19.1
Uninstalling numpy-1.19.1:
Successfully uninstalled numpy-1.19.1
Attempting uninstall: scipy
Found existing installation: scipy 1.4.1
Uninstalling scipy-1.4.1:
Successfully uninstalled scipy-1.4.1
Attempting uninstall: pyarrow
Found existing installation: pyarrow 6.0.1
Uninstalling pyarrow-6.0.1:
Successfully uninstalled pyarrow-6.0.1
Attempting uninstall: importlib-metadata
Found existing installation: importlib-metadata 4.8.2
Uninstalling importlib-metadata-4.8.2:
Successfully uninstalled importlib-metadata-4.8.2
Attempting uninstall: gluoncv
Found existing installation: gluoncv 0.8.0
Uninstalling gluoncv-0.8.0:
Successfully uninstalled gluoncv-0.8.0
Successfully installed Pillow-9.4.0 PyWavelets-1.3.0 absl-py-1.4.0 accelerate-0.13.2 aiohttp-3.8.4 aiosignal-1.3.1 albumentations-1.1.0 antlr4-python3-runtime-4.8 async-timeout-4.0.2 asynctest-0.13.0 autocfg-0.0.8 autogluon-0.6.2 autogluon.common-0.6.2 autogluon.core-0.6.2 autogluon.features-0.6.2 autogluon.multimodal-0.6.2 autogluon.tabular-0.6.2 autogluon.text-0.6.2 autogluon.timeseries-0.6.2 autogluon.vision-0.6.2 blis-0.7.9 cachetools-5.3.1 catalogue-2.0.8 catboost-1.1.1 charset-normalizer-3.1.0 click-8.0.4 confection-0.0.4 cymem-2.0.7 dask-2021.11.2 datasets-2.12.0 defusedxml-0.7.1 deprecated-1.2.14 distlib-0.3.6 distributed-2021.11.2 evaluate-0.3.0 fairscale-0.4.6 fastai-2.7.12 fastcore-1.5.29 fastdownload-0.0.7 fastprogress-1.0.3 filelock-3.12.1 frozenlist-1.3.3 future-0.18.3 gluoncv-0.10.5.post0 gluonts-0.11.12 google-auth-2.19.1 google-auth-oauthlib-0.4.6 grpcio-1.43.0 heapdict-1.0.1 huggingface-hub-0.15.1 hyperopt-0.2.7 importlib-metadata-6.6.0 importlib-resources-5.12.0 jsonschema-4.8.0 langcodes-3.3.0 lightgbm-3.3.5 locket-1.0.0 markdown-3.4.3 markdown-it-py-2.2.0 mdurl-0.1.2 model-index-0.1.11 msgpack-1.0.5 multidict-6.0.4 murmurhash-1.0.9 nlpaug-1.1.10 nltk-3.8.1 nptyping-1.4.4 numpy-1.21.6 oauthlib-3.2.2 omegaconf-2.1.2 opencv-python-headless-4.7.0.72 openmim-0.2.1 ordered-set-4.1.0 partd-1.4.0 pathy-0.10.1 patsy-0.5.3 platformdirs-3.1.1 pmdarima-1.8.5 preshed-3.0.8 py4j-0.10.9.7 pyDeprecate-0.3.2 pyarrow-12.0.0 pyasn1-modules-0.3.0 pydantic-1.10.9 pyrsistent-0.19.3 pytorch-lightning-1.7.7 pytorch-metric-learning-1.3.2 qudida-0.0.4 ray-2.0.1 regex-2023.6.3 requests-oauthlib-1.3.1 responses-0.18.0 rich-13.4.1 scikit-image-0.19.3 scipy-1.7.3 sentencepiece-0.1.99 seqeval-1.2.2 sktime-0.13.4 smart-open-5.2.1 sortedcontainers-2.4.0 spacy-3.5.3 spacy-legacy-3.0.12 spacy-loggers-1.0.4 srsly-2.4.6 statsmodels-0.13.5 tbats-1.1.3 tblib-1.7.0 tensorboard-2.11.2 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorboardX-2.6 text-unidecode-1.3 thinc-8.1.10 tifffile-2021.11.2 timm-0.6.13 tokenizers-0.13.3 toolz-0.12.0 torch-1.12.1 torchmetrics-0.8.2 torchtext-0.13.1 torchvision-0.13.1 tqdm-4.65.0 transformers-4.23.1 typer-0.7.0 typing-extensions-4.4.0 typish-1.9.3 virtualenv-20.21.1 wasabi-1.1.2 wrapt-1.15.0 xgboost-1.6.2 xxhash-3.2.0 yacs-0.1.8 yarl-1.9.2 zict-2.2.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
# create the .kaggle directory and an empty kaggle.json file
!mkdir -p /root/.kaggle
!touch /root/.kaggle/kaggle.json
!chmod 600 /root/.kaggle/kaggle.json
# Fill in your user name and key from creating the kaggle account and API token file
import json
kaggle_username = "lionelchong"
kaggle_key = "Key" # Key is stored in an .env variable
# Save API token the kaggle.json file
with open("/root/.kaggle/kaggle.json", "w") as f:
f.write(json.dumps({"username": kaggle_username, "key": kaggle_key}))
!pip install kaggle
# Download the dataset, it will be in a .zip file so you'll need to unzip it as well.
!kaggle competitions download -c bike-sharing-demand
# If you already downloaded it you can use the -o command to overwrite the file
!unzip -o bike-sharing-demand.zip
Requirement already satisfied: kaggle in /usr/local/lib/python3.7/site-packages (1.5.13) Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.16.0) Requirement already satisfied: certifi in /usr/local/lib/python3.7/site-packages (from kaggle) (2021.10.8) Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/site-packages (from kaggle) (2.8.0) Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from kaggle) (2.22.0) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/site-packages (from kaggle) (4.65.0) Requirement already satisfied: python-slugify in /usr/local/lib/python3.7/site-packages (from kaggle) (8.0.1) Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.25.11) Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.7/site-packages (from python-slugify->kaggle) (1.3) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (3.0.4) Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (2.8) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Downloading bike-sharing-demand.zip to /root/nd009t-c1-intro-to-ml-templates/cd0385-project-starter/project 0%| | 0.00/189k [00:00<?, ?B/s] 100%|████████████████████████████████████████| 189k/189k [00:00<00:00, 8.07MB/s] Archive: bike-sharing-demand.zip inflating: sampleSubmission.csv inflating: test.csv inflating: train.csv
import pandas as pd
from autogluon.tabular import TabularPredictor
# Create the train dataset in pandas by reading the csv
# Set the parsing of the datetime column so you can use some of the `dt` features in pandas later
train = pd.read_csv('train.csv')
train.head()
| datetime | season | holiday | workingday | weather | temp | atemp | humidity | windspeed | casual | registered | count | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2011-01-01 00:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 81 | 0.0 | 3 | 13 | 16 |
| 1 | 2011-01-01 01:00:00 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 8 | 32 | 40 |
| 2 | 2011-01-01 02:00:00 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 5 | 27 | 32 |
| 3 | 2011-01-01 03:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 3 | 10 | 13 |
| 4 | 2011-01-01 04:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 0 | 1 | 1 |
# Simple output of the train dataset to view some of the min/max/varition of the dataset features.
# Create the test pandas dataframe in pandas by reading the csv, remember to parse the datetime!
test = pd.read_csv('test.csv')
test.head()
| datetime | season | holiday | workingday | weather | temp | atemp | humidity | windspeed | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 2011-01-20 00:00:00 | 1 | 0 | 1 | 1 | 10.66 | 11.365 | 56 | 26.0027 |
| 1 | 2011-01-20 01:00:00 | 1 | 0 | 1 | 1 | 10.66 | 13.635 | 56 | 0.0000 |
| 2 | 2011-01-20 02:00:00 | 1 | 0 | 1 | 1 | 10.66 | 13.635 | 56 | 0.0000 |
| 3 | 2011-01-20 03:00:00 | 1 | 0 | 1 | 1 | 10.66 | 12.880 | 56 | 11.0014 |
| 4 | 2011-01-20 04:00:00 | 1 | 0 | 1 | 1 | 10.66 | 12.880 | 56 | 11.0014 |
# Same thing as train and test dataset
submission = pd.read_csv('sampleSubmission.csv')
submission.head()
| datetime | count | |
|---|---|---|
| 0 | 2011-01-20 00:00:00 | 0 |
| 1 | 2011-01-20 01:00:00 | 0 |
| 2 | 2011-01-20 02:00:00 | 0 |
| 3 | 2011-01-20 03:00:00 | 0 |
| 4 | 2011-01-20 04:00:00 | 0 |
Requirements:
count, so it is the label we are setting.casual and registered columns as they are also not present in the test dataset. root_mean_squared_error as the metric to use for evaluation.best_quality to focus on creating the best model.predictor = TabularPredictor(
label="count", problem_type="regression", eval_metric="rmse"
).fit(
train_data=train.drop(['casual', 'registered'], axis=1),
time_limit=600,
presets='best_quality')
No path specified. Models will be saved in: "AutogluonModels/ag-20230611_151853/"
Presets specified: ['best_quality']
Stack configuration (auto_stack=True): num_stack_levels=1, num_bag_folds=8, num_bag_sets=20
Beginning AutoGluon training ... Time limit = 600s
AutoGluon will save models to "AutogluonModels/ag-20230611_151853/"
AutoGluon Version: 0.6.2
Python Version: 3.7.10
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Thu May 4 09:55:30 UTC 2023
Train Data Rows: 10886
Train Data Columns: 9
Label Column: count
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 2922.18 MB
Train Data (Original) Memory Usage: 1.52 MB (0.1% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 2 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting DatetimeFeatureGenerator...
/usr/local/lib/python3.7/site-packages/autogluon/features/generators/datetime.py:59: FutureWarning: casting datetime64[ns, UTC] values to int64 with .astype(...) is deprecated and will raise in a future version. Use .view(...) instead.
good_rows = series[~series.isin(bad_rows)].astype(np.int64)
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 5 | ['season', 'holiday', 'workingday', 'weather', 'humidity']
('object', ['datetime_as_object']) : 1 | ['datetime']
Types of features in processed data (raw dtype, special dtypes):
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 3 | ['season', 'weather', 'humidity']
('int', ['bool']) : 2 | ['holiday', 'workingday']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
0.4s = Fit runtime
9 features in original data used to generate 13 features in processed data.
Train Data (Processed) Memory Usage: 0.98 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.43s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
This metric's sign has been flipped to adhere to being higher_is_better. The metric score can be multiplied by -1 to get the metric value.
To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 11 L1 models ...
Fitting model: KNeighborsUnif_BAG_L1 ... Training model for up to 399.61s of the 599.56s of remaining time.
-101.5462 = Validation score (-root_mean_squared_error)
0.06s = Training runtime
0.11s = Validation runtime
Fitting model: KNeighborsDist_BAG_L1 ... Training model for up to 395.19s of the 595.14s of remaining time.
-84.1251 = Validation score (-root_mean_squared_error)
0.03s = Training runtime
0.1s = Validation runtime
Fitting model: LightGBMXT_BAG_L1 ... Training model for up to 394.83s of the 594.78s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-131.4609 = Validation score (-root_mean_squared_error)
66.6s = Training runtime
7.24s = Validation runtime
Fitting model: LightGBM_BAG_L1 ... Training model for up to 318.35s of the 518.3s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-131.0542 = Validation score (-root_mean_squared_error)
29.55s = Training runtime
1.38s = Validation runtime
Fitting model: RandomForestMSE_BAG_L1 ... Training model for up to 284.64s of the 484.6s of remaining time.
-116.5443 = Validation score (-root_mean_squared_error)
10.96s = Training runtime
0.6s = Validation runtime
Fitting model: CatBoost_BAG_L1 ... Training model for up to 270.4s of the 470.35s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-130.5332 = Validation score (-root_mean_squared_error)
200.71s = Training runtime
0.1s = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L1 ... Training model for up to 65.85s of the 265.8s of remaining time.
-124.5881 = Validation score (-root_mean_squared_error)
4.94s = Training runtime
0.53s = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L1 ... Training model for up to 57.74s of the 257.69s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-138.2085 = Validation score (-root_mean_squared_error)
72.66s = Training runtime
0.51s = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 181.14s of remaining time.
-84.1251 = Validation score (-root_mean_squared_error)
0.64s = Training runtime
0.0s = Validation runtime
Fitting 9 L2 models ...
Fitting model: LightGBMXT_BAG_L2 ... Training model for up to 180.41s of the 180.39s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-60.401 = Validation score (-root_mean_squared_error)
53.89s = Training runtime
3.08s = Validation runtime
Fitting model: LightGBM_BAG_L2 ... Training model for up to 121.26s of the 121.23s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-55.1131 = Validation score (-root_mean_squared_error)
25.57s = Training runtime
0.23s = Validation runtime
Fitting model: RandomForestMSE_BAG_L2 ... Training model for up to 91.81s of the 91.79s of remaining time.
-53.407 = Validation score (-root_mean_squared_error)
26.8s = Training runtime
0.61s = Validation runtime
Fitting model: CatBoost_BAG_L2 ... Training model for up to 61.92s of the 61.9s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-55.6347 = Validation score (-root_mean_squared_error)
61.99s = Training runtime
0.05s = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the -3.93s of remaining time.
-53.0925 = Validation score (-root_mean_squared_error)
0.33s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 604.49s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20230611_151853/")
predictor.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
model score_val pred_time_val fit_time pred_time_val_marginal fit_time_marginal stack_level can_infer fit_order
0 WeightedEnsemble_L3 -53.092490 14.530773 554.111094 0.001197 0.330786 3 True 14
1 RandomForestMSE_BAG_L2 -53.407033 11.173833 412.322954 0.608464 26.798874 2 True 12
2 LightGBM_BAG_L2 -55.113121 10.790813 411.092591 0.225444 25.568511 2 True 11
3 CatBoost_BAG_L2 -55.634739 10.619263 447.518704 0.053894 61.994624 2 True 13
4 LightGBMXT_BAG_L2 -60.401023 13.641774 439.418299 3.076405 53.894219 2 True 10
5 KNeighborsDist_BAG_L1 -84.125061 0.104620 0.029401 0.104620 0.029401 1 True 2
6 WeightedEnsemble_L2 -84.125061 0.105885 0.668792 0.001266 0.639390 2 True 9
7 KNeighborsUnif_BAG_L1 -101.546199 0.106041 0.063636 0.106041 0.063636 1 True 1
8 RandomForestMSE_BAG_L1 -116.544294 0.600751 10.962398 0.600751 10.962398 1 True 5
9 ExtraTreesMSE_BAG_L1 -124.588053 0.526679 4.943228 0.526679 4.943228 1 True 7
10 CatBoost_BAG_L1 -130.533194 0.100010 200.710994 0.100010 200.710994 1 True 6
11 LightGBM_BAG_L1 -131.054162 1.375981 29.551827 1.375981 29.551827 1 True 4
12 LightGBMXT_BAG_L1 -131.460909 7.240988 66.602082 7.240988 66.602082 1 True 3
13 NeuralNetFastAI_BAG_L1 -138.208491 0.510299 72.660515 0.510299 72.660515 1 True 8
Number of models trained: 14
Types of models trained:
{'WeightedEnsembleModel', 'StackerEnsembleModel_KNN', 'StackerEnsembleModel_XT', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_NNFastAiTabular', 'StackerEnsembleModel_RF', 'StackerEnsembleModel_CatBoost'}
Bagging used: True (with 8 folds)
Multi-layer stack-ensembling used: True (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 3 | ['season', 'weather', 'humidity']
('int', ['bool']) : 2 | ['holiday', 'workingday']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
Plot summary of models saved to file: AutogluonModels/ag-20230611_151853/SummaryOfModels.html
*** End of fit() summary ***
{'model_types': {'KNeighborsUnif_BAG_L1': 'StackerEnsembleModel_KNN',
'KNeighborsDist_BAG_L1': 'StackerEnsembleModel_KNN',
'LightGBMXT_BAG_L1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L1': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L1': 'StackerEnsembleModel_CatBoost',
'ExtraTreesMSE_BAG_L1': 'StackerEnsembleModel_XT',
'NeuralNetFastAI_BAG_L1': 'StackerEnsembleModel_NNFastAiTabular',
'WeightedEnsemble_L2': 'WeightedEnsembleModel',
'LightGBMXT_BAG_L2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L2': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L2': 'StackerEnsembleModel_CatBoost',
'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
'model_performance': {'KNeighborsUnif_BAG_L1': -101.54619908446061,
'KNeighborsDist_BAG_L1': -84.12506123181602,
'LightGBMXT_BAG_L1': -131.46090891834504,
'LightGBM_BAG_L1': -131.054161598899,
'RandomForestMSE_BAG_L1': -116.54429428704391,
'CatBoost_BAG_L1': -130.5331939673838,
'ExtraTreesMSE_BAG_L1': -124.58805258915959,
'NeuralNetFastAI_BAG_L1': -138.2084908141198,
'WeightedEnsemble_L2': -84.12506123181602,
'LightGBMXT_BAG_L2': -60.40102314592625,
'LightGBM_BAG_L2': -55.11312122378169,
'RandomForestMSE_BAG_L2': -53.40703329979414,
'CatBoost_BAG_L2': -55.63473865135565,
'WeightedEnsemble_L3': -53.09249015726556},
'model_best': 'WeightedEnsemble_L3',
'model_paths': {'KNeighborsUnif_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/KNeighborsUnif_BAG_L1/',
'KNeighborsDist_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/KNeighborsDist_BAG_L1/',
'LightGBMXT_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/LightGBMXT_BAG_L1/',
'LightGBM_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/LightGBM_BAG_L1/',
'RandomForestMSE_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/RandomForestMSE_BAG_L1/',
'CatBoost_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/CatBoost_BAG_L1/',
'ExtraTreesMSE_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/ExtraTreesMSE_BAG_L1/',
'NeuralNetFastAI_BAG_L1': 'AutogluonModels/ag-20230611_151853/models/NeuralNetFastAI_BAG_L1/',
'WeightedEnsemble_L2': 'AutogluonModels/ag-20230611_151853/models/WeightedEnsemble_L2/',
'LightGBMXT_BAG_L2': 'AutogluonModels/ag-20230611_151853/models/LightGBMXT_BAG_L2/',
'LightGBM_BAG_L2': 'AutogluonModels/ag-20230611_151853/models/LightGBM_BAG_L2/',
'RandomForestMSE_BAG_L2': 'AutogluonModels/ag-20230611_151853/models/RandomForestMSE_BAG_L2/',
'CatBoost_BAG_L2': 'AutogluonModels/ag-20230611_151853/models/CatBoost_BAG_L2/',
'WeightedEnsemble_L3': 'AutogluonModels/ag-20230611_151853/models/WeightedEnsemble_L3/'},
'model_fit_times': {'KNeighborsUnif_BAG_L1': 0.06363558769226074,
'KNeighborsDist_BAG_L1': 0.029401063919067383,
'LightGBMXT_BAG_L1': 66.6020815372467,
'LightGBM_BAG_L1': 29.55182695388794,
'RandomForestMSE_BAG_L1': 10.962397575378418,
'CatBoost_BAG_L1': 200.71099424362183,
'ExtraTreesMSE_BAG_L1': 4.943228483200073,
'NeuralNetFastAI_BAG_L1': 72.66051483154297,
'WeightedEnsemble_L2': 0.6393904685974121,
'LightGBMXT_BAG_L2': 53.89421892166138,
'LightGBM_BAG_L2': 25.56851100921631,
'RandomForestMSE_BAG_L2': 26.79887366294861,
'CatBoost_BAG_L2': 61.99462413787842,
'WeightedEnsemble_L3': 0.33078551292419434},
'model_pred_times': {'KNeighborsUnif_BAG_L1': 0.10604143142700195,
'KNeighborsDist_BAG_L1': 0.10461950302124023,
'LightGBMXT_BAG_L1': 7.240987539291382,
'LightGBM_BAG_L1': 1.375981092453003,
'RandomForestMSE_BAG_L1': 0.6007513999938965,
'CatBoost_BAG_L1': 0.10001039505004883,
'ExtraTreesMSE_BAG_L1': 0.5266790390014648,
'NeuralNetFastAI_BAG_L1': 0.5102987289428711,
'WeightedEnsemble_L2': 0.0012657642364501953,
'LightGBMXT_BAG_L2': 3.0764048099517822,
'LightGBM_BAG_L2': 0.22544360160827637,
'RandomForestMSE_BAG_L2': 0.6084635257720947,
'CatBoost_BAG_L2': 0.0538942813873291,
'WeightedEnsemble_L3': 0.0011973381042480469},
'num_bag_folds': 8,
'max_stack_level': 3,
'model_hyperparams': {'KNeighborsUnif_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'KNeighborsDist_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'LightGBMXT_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'ExtraTreesMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'NeuralNetFastAI_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L2': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBMXT_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L3': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True}},
'leaderboard': model score_val pred_time_val fit_time \
0 WeightedEnsemble_L3 -53.092490 14.530773 554.111094
1 RandomForestMSE_BAG_L2 -53.407033 11.173833 412.322954
2 LightGBM_BAG_L2 -55.113121 10.790813 411.092591
3 CatBoost_BAG_L2 -55.634739 10.619263 447.518704
4 LightGBMXT_BAG_L2 -60.401023 13.641774 439.418299
5 KNeighborsDist_BAG_L1 -84.125061 0.104620 0.029401
6 WeightedEnsemble_L2 -84.125061 0.105885 0.668792
7 KNeighborsUnif_BAG_L1 -101.546199 0.106041 0.063636
8 RandomForestMSE_BAG_L1 -116.544294 0.600751 10.962398
9 ExtraTreesMSE_BAG_L1 -124.588053 0.526679 4.943228
10 CatBoost_BAG_L1 -130.533194 0.100010 200.710994
11 LightGBM_BAG_L1 -131.054162 1.375981 29.551827
12 LightGBMXT_BAG_L1 -131.460909 7.240988 66.602082
13 NeuralNetFastAI_BAG_L1 -138.208491 0.510299 72.660515
pred_time_val_marginal fit_time_marginal stack_level can_infer \
0 0.001197 0.330786 3 True
1 0.608464 26.798874 2 True
2 0.225444 25.568511 2 True
3 0.053894 61.994624 2 True
4 3.076405 53.894219 2 True
5 0.104620 0.029401 1 True
6 0.001266 0.639390 2 True
7 0.106041 0.063636 1 True
8 0.600751 10.962398 1 True
9 0.526679 4.943228 1 True
10 0.100010 200.710994 1 True
11 1.375981 29.551827 1 True
12 7.240988 66.602082 1 True
13 0.510299 72.660515 1 True
fit_order
0 14
1 12
2 11
3 13
4 10
5 2
6 9
7 1
8 5
9 7
10 6
11 4
12 3
13 8 }
predictions = pd.DataFrame(data={'datetime': test['datetime'], 'Pred_count': predictor.predict(test)})
predictions.head()
| datetime | Pred_count | |
|---|---|---|
| 0 | 2011-01-20 00:00:00 | 24.123066 |
| 1 | 2011-01-20 01:00:00 | 42.460495 |
| 2 | 2011-01-20 02:00:00 | 46.532757 |
| 3 | 2011-01-20 03:00:00 | 50.135544 |
| 4 | 2011-01-20 04:00:00 | 52.505470 |
# Describe the `predictions` series to see if there are any negative values
predictions.describe()
| Pred_count | |
|---|---|
| count | 6493.000000 |
| mean | 100.627243 |
| std | 89.699265 |
| min | 3.202173 |
| 25% | 20.407969 |
| 50% | 64.216888 |
| 75% | 167.488281 |
| max | 364.660767 |
# How many negative values do we have?
def negative_function(value):
return value[value < 0].sum()
negative_values = predictions.groupby(predictions['Pred_count'])
print(negative_values['Pred_count'].agg([('negcount', negative_function)]))
negcount Pred_count 3.202173 0.0 3.227723 0.0 3.231407 0.0 3.265340 0.0 3.364769 0.0 ... ... 362.632050 0.0 363.546387 0.0 364.136047 0.0 364.237183 0.0 364.660767 0.0 [6187 rows x 1 columns]
# Set them to zero
predictions[predictions['Pred_count']<0] = 0
# Check predictions after setting them to zero
predictions.describe()
| Pred_count | |
|---|---|
| count | 6493.000000 |
| mean | 100.627243 |
| std | 89.699265 |
| min | 3.202173 |
| 25% | 20.407969 |
| 50% | 64.216888 |
| 75% | 167.488281 |
| max | 364.660767 |
submission["count"] = predictions['Pred_count']
submission.to_csv("submission.csv", index=False)
!pip install -U kaggle
!kaggle competitions submit -c bike-sharing-demand -f submission.csv -m "first raw submission"
Collecting kaggle
Downloading kaggle-1.5.13.tar.gz (63 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.3/63.3 kB 1.8 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.16.0)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/site-packages (from kaggle) (2021.10.8)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/site-packages (from kaggle) (2.8.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from kaggle) (2.22.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/site-packages (from kaggle) (4.65.0)
Collecting python-slugify (from kaggle)
Downloading python_slugify-8.0.1-py2.py3-none-any.whl (9.7 kB)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.25.11)
Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.7/site-packages (from python-slugify->kaggle) (1.3)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (2.8)
Building wheels for collected packages: kaggle
Building wheel for kaggle (setup.py) ... done
Created wheel for kaggle: filename=kaggle-1.5.13-py3-none-any.whl size=77717 sha256=5db9d2c4a400005ee6a4c1963c2f24603ed18449735ac6319dd763ef590a42a9
Stored in directory: /root/.cache/pip/wheels/fd/97/a6/3372cb23468915cbcf108338dd29c73379fd1a55828ec608ba
Successfully built kaggle
Installing collected packages: python-slugify, kaggle
Successfully installed kaggle-1.5.13 python-slugify-8.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
100%|█████████████████████████████████████████| 188k/188k [00:00<00:00, 354kB/s]
Successfully submitted to Bike Sharing Demand
My Submissions¶!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName date description status publicScore privateScore -------------- ------------------- -------------------- -------- ----------- ------------ submission.csv 2023-06-11 15:29:46 first raw submission complete 1.79005 1.79005
1.79005¶# Create a histogram of all features to show the distribution of each one relative to the data. This is part of the exploritory data analysis
train.hist()
array([[<AxesSubplot:title={'center':'season'}>,
<AxesSubplot:title={'center':'holiday'}>,
<AxesSubplot:title={'center':'workingday'}>],
[<AxesSubplot:title={'center':'weather'}>,
<AxesSubplot:title={'center':'temp'}>,
<AxesSubplot:title={'center':'atemp'}>],
[<AxesSubplot:title={'center':'humidity'}>,
<AxesSubplot:title={'center':'windspeed'}>,
<AxesSubplot:title={'center':'casual'}>],
[<AxesSubplot:title={'center':'registered'}>,
<AxesSubplot:title={'center':'count'}>, <AxesSubplot:>]],
dtype=object)
# create a new feature
# Break datetime into year, month, day and hour
train.loc[:, "datetime"] = pd.to_datetime(train.loc[:, "datetime"])
test.loc[:, "datetime"] = pd.to_datetime(test.loc[:, "datetime"])
train['year'] = train['datetime'].dt.year
train['month'] = train['datetime'].dt.month
train['day'] = train['datetime'].dt.day
train['hour'] = train['datetime'].dt.hour
test['year'] = test['datetime'].dt.year
test['month'] = test['datetime'].dt.month
test['day'] = test['datetime'].dt.day
test['hour'] = test['datetime'].dt.hour
train["season"] = train["season"].astype("category")
train["weather"] = train["weather"].astype("category")
test["season"] = test["season"].astype("category")
test["weather"] = test["weather"].astype("category")
# View are new feature
train.head()
| datetime | season | holiday | workingday | weather | temp | atemp | humidity | windspeed | casual | registered | count | year | month | day | hour | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2011-01-01 00:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 81 | 0.0 | 3 | 13 | 16 | 2011 | 1 | 1 | 0 |
| 1 | 2011-01-01 01:00:00 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 8 | 32 | 40 | 2011 | 1 | 1 | 1 |
| 2 | 2011-01-01 02:00:00 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 5 | 27 | 32 | 2011 | 1 | 1 | 2 |
| 3 | 2011-01-01 03:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 3 | 10 | 13 | 2011 | 1 | 1 | 3 |
| 4 | 2011-01-01 04:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 0 | 1 | 1 | 2011 | 1 | 1 | 4 |
# View histogram of all features again now with the hour feature
train.hist()
array([[<AxesSubplot:title={'center':'datetime'}>,
<AxesSubplot:title={'center':'holiday'}>,
<AxesSubplot:title={'center':'workingday'}>,
<AxesSubplot:title={'center':'temp'}>],
[<AxesSubplot:title={'center':'atemp'}>,
<AxesSubplot:title={'center':'humidity'}>,
<AxesSubplot:title={'center':'windspeed'}>,
<AxesSubplot:title={'center':'casual'}>],
[<AxesSubplot:title={'center':'registered'}>,
<AxesSubplot:title={'center':'count'}>,
<AxesSubplot:title={'center':'year'}>,
<AxesSubplot:title={'center':'month'}>],
[<AxesSubplot:title={'center':'day'}>,
<AxesSubplot:title={'center':'hour'}>, <AxesSubplot:>,
<AxesSubplot:>]], dtype=object)
predictor_new_features = TabularPredictor(
label="count", problem_type="regression", eval_metric="rmse"
).fit(
train_data=train.drop(['casual', 'registered'], axis=1),
time_limit=600,
presets='best_quality')
No path specified. Models will be saved in: "AutogluonModels/ag-20230611_154539/"
Presets specified: ['best_quality']
Stack configuration (auto_stack=True): num_stack_levels=1, num_bag_folds=8, num_bag_sets=20
Beginning AutoGluon training ... Time limit = 600s
AutoGluon will save models to "AutogluonModels/ag-20230611_154539/"
AutoGluon Version: 0.6.2
Python Version: 3.7.10
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Thu May 4 09:55:30 UTC 2023
Train Data Rows: 10886
Train Data Columns: 13
Label Column: count
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 1966.6 MB
Train Data (Original) Memory Usage: 0.98 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Fitting DatetimeFeatureGenerator...
/usr/local/lib/python3.7/site-packages/autogluon/features/generators/datetime.py:59: FutureWarning: casting datetime64[ns, UTC] values to int64 with .astype(...) is deprecated and will raise in a future version. Use .view(...) instead.
good_rows = series[~series.isin(bad_rows)].astype(np.int64)
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('datetime', []) : 1 | ['datetime']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 7 | ['holiday', 'workingday', 'humidity', 'year', 'month', ...]
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
0.6s = Fit runtime
13 features in original data used to generate 17 features in processed data.
Train Data (Processed) Memory Usage: 1.1 MB (0.1% of available memory)
Data preprocessing and feature engineering runtime = 0.7s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
This metric's sign has been flipped to adhere to being higher_is_better. The metric score can be multiplied by -1 to get the metric value.
To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 11 L1 models ...
Fitting model: KNeighborsUnif_BAG_L1 ... Training model for up to 399.43s of the 599.29s of remaining time.
-101.5462 = Validation score (-root_mean_squared_error)
0.04s = Training runtime
0.1s = Validation runtime
Fitting model: KNeighborsDist_BAG_L1 ... Training model for up to 399.07s of the 598.93s of remaining time.
-84.1251 = Validation score (-root_mean_squared_error)
0.03s = Training runtime
0.1s = Validation runtime
Fitting model: LightGBMXT_BAG_L1 ... Training model for up to 398.7s of the 598.57s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-34.346 = Validation score (-root_mean_squared_error)
86.46s = Training runtime
9.88s = Validation runtime
Fitting model: LightGBM_BAG_L1 ... Training model for up to 307.04s of the 506.9s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-33.9173 = Validation score (-root_mean_squared_error)
44.2s = Training runtime
3.56s = Validation runtime
Fitting model: RandomForestMSE_BAG_L1 ... Training model for up to 258.08s of the 457.94s of remaining time.
-38.3149 = Validation score (-root_mean_squared_error)
14.47s = Training runtime
0.58s = Validation runtime
Fitting model: CatBoost_BAG_L1 ... Training model for up to 240.51s of the 440.37s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-33.9718 = Validation score (-root_mean_squared_error)
205.4s = Training runtime
0.17s = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L1 ... Training model for up to 30.99s of the 230.85s of remaining time.
-38.2952 = Validation score (-root_mean_squared_error)
6.65s = Training runtime
0.56s = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L1 ... Training model for up to 21.27s of the 221.13s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-103.8428 = Validation score (-root_mean_squared_error)
43.37s = Training runtime
0.49s = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 173.84s of remaining time.
-32.1324 = Validation score (-root_mean_squared_error)
0.67s = Training runtime
0.0s = Validation runtime
Fitting 9 L2 models ...
Fitting model: LightGBMXT_BAG_L2 ... Training model for up to 173.09s of the 173.06s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-31.1799 = Validation score (-root_mean_squared_error)
33.72s = Training runtime
1.07s = Validation runtime
Fitting model: LightGBM_BAG_L2 ... Training model for up to 134.32s of the 134.29s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-30.4233 = Validation score (-root_mean_squared_error)
25.88s = Training runtime
0.29s = Validation runtime
Fitting model: RandomForestMSE_BAG_L2 ... Training model for up to 104.36s of the 104.33s of remaining time.
-31.5082 = Validation score (-root_mean_squared_error)
31.83s = Training runtime
0.63s = Validation runtime
Fitting model: CatBoost_BAG_L2 ... Training model for up to 69.61s of the 69.58s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-30.3736 = Validation score (-root_mean_squared_error)
67.87s = Training runtime
0.11s = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the -2.28s of remaining time.
-30.0844 = Validation score (-root_mean_squared_error)
0.29s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 602.77s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20230611_154539/")
predictor_new_features.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
model score_val pred_time_val fit_time pred_time_val_marginal fit_time_marginal stack_level can_infer fit_order
0 WeightedEnsemble_L3 -30.084351 16.909217 528.380250 0.000771 0.288950 3 True 14
1 CatBoost_BAG_L2 -30.373557 15.544385 468.489517 0.109781 67.872453 2 True 13
2 LightGBM_BAG_L2 -30.423259 15.729353 426.498674 0.294749 25.881609 2 True 11
3 LightGBMXT_BAG_L2 -31.179943 16.503916 434.337237 1.069312 33.720173 2 True 10
4 RandomForestMSE_BAG_L2 -31.508185 16.060942 432.443680 0.626337 31.826616 2 True 12
5 WeightedEnsemble_L2 -32.132426 14.285585 351.234542 0.001356 0.670159 2 True 9
6 LightGBM_BAG_L1 -33.917339 3.559763 44.200528 3.559763 44.200528 1 True 4
7 CatBoost_BAG_L1 -33.971769 0.170114 205.399064 0.170114 205.399064 1 True 6
8 LightGBMXT_BAG_L1 -34.345997 9.875449 86.456437 9.875449 86.456437 1 True 3
9 ExtraTreesMSE_BAG_L1 -38.295243 0.560143 6.649233 0.560143 6.649233 1 True 7
10 RandomForestMSE_BAG_L1 -38.314947 0.575209 14.474546 0.575209 14.474546 1 True 5
11 KNeighborsDist_BAG_L1 -84.125061 0.103694 0.033808 0.103694 0.033808 1 True 2
12 KNeighborsUnif_BAG_L1 -101.546199 0.103020 0.035200 0.103020 0.035200 1 True 1
13 NeuralNetFastAI_BAG_L1 -103.842847 0.487212 43.368248 0.487212 43.368248 1 True 8
Number of models trained: 14
Types of models trained:
{'WeightedEnsembleModel', 'StackerEnsembleModel_KNN', 'StackerEnsembleModel_XT', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_NNFastAiTabular', 'StackerEnsembleModel_RF', 'StackerEnsembleModel_CatBoost'}
Bagging used: True (with 8 folds)
Multi-layer stack-ensembling used: True (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
Plot summary of models saved to file: AutogluonModels/ag-20230611_154539/SummaryOfModels.html
*** End of fit() summary ***
{'model_types': {'KNeighborsUnif_BAG_L1': 'StackerEnsembleModel_KNN',
'KNeighborsDist_BAG_L1': 'StackerEnsembleModel_KNN',
'LightGBMXT_BAG_L1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L1': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L1': 'StackerEnsembleModel_CatBoost',
'ExtraTreesMSE_BAG_L1': 'StackerEnsembleModel_XT',
'NeuralNetFastAI_BAG_L1': 'StackerEnsembleModel_NNFastAiTabular',
'WeightedEnsemble_L2': 'WeightedEnsembleModel',
'LightGBMXT_BAG_L2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L2': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L2': 'StackerEnsembleModel_CatBoost',
'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
'model_performance': {'KNeighborsUnif_BAG_L1': -101.54619908446061,
'KNeighborsDist_BAG_L1': -84.12506123181602,
'LightGBMXT_BAG_L1': -34.34599701170154,
'LightGBM_BAG_L1': -33.91733862651761,
'RandomForestMSE_BAG_L1': -38.31494666065859,
'CatBoost_BAG_L1': -33.97176883234122,
'ExtraTreesMSE_BAG_L1': -38.29524277410234,
'NeuralNetFastAI_BAG_L1': -103.842846933631,
'WeightedEnsemble_L2': -32.13242584358949,
'LightGBMXT_BAG_L2': -31.179942907396665,
'LightGBM_BAG_L2': -30.423258655809637,
'RandomForestMSE_BAG_L2': -31.508184844819915,
'CatBoost_BAG_L2': -30.37355681969245,
'WeightedEnsemble_L3': -30.08435065702291},
'model_best': 'WeightedEnsemble_L3',
'model_paths': {'KNeighborsUnif_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/KNeighborsUnif_BAG_L1/',
'KNeighborsDist_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/KNeighborsDist_BAG_L1/',
'LightGBMXT_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/LightGBMXT_BAG_L1/',
'LightGBM_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/LightGBM_BAG_L1/',
'RandomForestMSE_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/RandomForestMSE_BAG_L1/',
'CatBoost_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/CatBoost_BAG_L1/',
'ExtraTreesMSE_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/ExtraTreesMSE_BAG_L1/',
'NeuralNetFastAI_BAG_L1': 'AutogluonModels/ag-20230611_154539/models/NeuralNetFastAI_BAG_L1/',
'WeightedEnsemble_L2': 'AutogluonModels/ag-20230611_154539/models/WeightedEnsemble_L2/',
'LightGBMXT_BAG_L2': 'AutogluonModels/ag-20230611_154539/models/LightGBMXT_BAG_L2/',
'LightGBM_BAG_L2': 'AutogluonModels/ag-20230611_154539/models/LightGBM_BAG_L2/',
'RandomForestMSE_BAG_L2': 'AutogluonModels/ag-20230611_154539/models/RandomForestMSE_BAG_L2/',
'CatBoost_BAG_L2': 'AutogluonModels/ag-20230611_154539/models/CatBoost_BAG_L2/',
'WeightedEnsemble_L3': 'AutogluonModels/ag-20230611_154539/models/WeightedEnsemble_L3/'},
'model_fit_times': {'KNeighborsUnif_BAG_L1': 0.03520035743713379,
'KNeighborsDist_BAG_L1': 0.033808231353759766,
'LightGBMXT_BAG_L1': 86.4564368724823,
'LightGBM_BAG_L1': 44.200528144836426,
'RandomForestMSE_BAG_L1': 14.47454571723938,
'CatBoost_BAG_L1': 205.39906406402588,
'ExtraTreesMSE_BAG_L1': 6.649233102798462,
'NeuralNetFastAI_BAG_L1': 43.368247747421265,
'WeightedEnsemble_L2': 0.6701593399047852,
'LightGBMXT_BAG_L2': 33.7201726436615,
'LightGBM_BAG_L2': 25.881609439849854,
'RandomForestMSE_BAG_L2': 31.826615810394287,
'CatBoost_BAG_L2': 67.87245321273804,
'WeightedEnsemble_L3': 0.28894996643066406},
'model_pred_times': {'KNeighborsUnif_BAG_L1': 0.10301971435546875,
'KNeighborsDist_BAG_L1': 0.10369372367858887,
'LightGBMXT_BAG_L1': 9.875449180603027,
'LightGBM_BAG_L1': 3.5597634315490723,
'RandomForestMSE_BAG_L1': 0.5752091407775879,
'CatBoost_BAG_L1': 0.17011356353759766,
'ExtraTreesMSE_BAG_L1': 0.560143232345581,
'NeuralNetFastAI_BAG_L1': 0.4872124195098877,
'WeightedEnsemble_L2': 0.0013561248779296875,
'LightGBMXT_BAG_L2': 1.0693118572235107,
'LightGBM_BAG_L2': 0.29474902153015137,
'RandomForestMSE_BAG_L2': 0.6263372898101807,
'CatBoost_BAG_L2': 0.10978102684020996,
'WeightedEnsemble_L3': 0.0007710456848144531},
'num_bag_folds': 8,
'max_stack_level': 3,
'model_hyperparams': {'KNeighborsUnif_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'KNeighborsDist_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'LightGBMXT_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'ExtraTreesMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'NeuralNetFastAI_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L2': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBMXT_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L3': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True}},
'leaderboard': model score_val pred_time_val fit_time \
0 WeightedEnsemble_L3 -30.084351 16.909217 528.380250
1 CatBoost_BAG_L2 -30.373557 15.544385 468.489517
2 LightGBM_BAG_L2 -30.423259 15.729353 426.498674
3 LightGBMXT_BAG_L2 -31.179943 16.503916 434.337237
4 RandomForestMSE_BAG_L2 -31.508185 16.060942 432.443680
5 WeightedEnsemble_L2 -32.132426 14.285585 351.234542
6 LightGBM_BAG_L1 -33.917339 3.559763 44.200528
7 CatBoost_BAG_L1 -33.971769 0.170114 205.399064
8 LightGBMXT_BAG_L1 -34.345997 9.875449 86.456437
9 ExtraTreesMSE_BAG_L1 -38.295243 0.560143 6.649233
10 RandomForestMSE_BAG_L1 -38.314947 0.575209 14.474546
11 KNeighborsDist_BAG_L1 -84.125061 0.103694 0.033808
12 KNeighborsUnif_BAG_L1 -101.546199 0.103020 0.035200
13 NeuralNetFastAI_BAG_L1 -103.842847 0.487212 43.368248
pred_time_val_marginal fit_time_marginal stack_level can_infer \
0 0.000771 0.288950 3 True
1 0.109781 67.872453 2 True
2 0.294749 25.881609 2 True
3 1.069312 33.720173 2 True
4 0.626337 31.826616 2 True
5 0.001356 0.670159 2 True
6 3.559763 44.200528 1 True
7 0.170114 205.399064 1 True
8 9.875449 86.456437 1 True
9 0.560143 6.649233 1 True
10 0.575209 14.474546 1 True
11 0.103694 0.033808 1 True
12 0.103020 0.035200 1 True
13 0.487212 43.368248 1 True
fit_order
0 14
1 13
2 11
3 10
4 12
5 9
6 4
7 6
8 3
9 7
10 5
11 2
12 1
13 8 }
# Use the new model to prefict using the test data
predictions_new_features = predictor_new_features.predict(test)
predictions_new_features = {'datetime': test['datetime'], 'Pred_count': predictions_new_features}
predictions_new_features = pd.DataFrame(data=predictions_new_features)
# Check the predictions head
predictions_new_features.head()
| datetime | Pred_count | |
|---|---|---|
| 0 | 2011-01-20 00:00:00 | 15.859232 |
| 1 | 2011-01-20 01:00:00 | 11.959195 |
| 2 | 2011-01-20 02:00:00 | 10.928431 |
| 3 | 2011-01-20 03:00:00 | 9.760633 |
| 4 | 2011-01-20 04:00:00 | 8.368550 |
# Remember to set all negative values to zero
predictions_new_features[predictions_new_features['Pred_count']<0] = 0
# Same submitting predictions
submission_new_features = pd.read_csv('submission.csv')
submission_new_features["count"] = predictions_new_features['Pred_count']
submission_new_features.to_csv("submission_new_features.csv", index=False)
!kaggle competitions submit -c bike-sharing-demand -f submission_new_features.csv -m "new features"
100%|█████████████████████████████████████████| 188k/188k [00:00<00:00, 368kB/s] Successfully submitted to Bike Sharing Demand
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName date description status publicScore privateScore --------------------------- ------------------- -------------------- -------- ----------- ------------ submission_new_features.csv 2023-06-11 15:56:21 new features complete 0.68994 0.68994 submission.csv 2023-06-11 15:29:46 first raw submission complete 1.79005 1.79005
0.68994¶hyperparameter and hyperparameter_tune_kwargs arguments.import autogluon.core as ag
# NN Dropout probability
nn_options = {
'dropout_prob': ag.space.Real(0.0, 0.5, default=0.1),
}
# No. of boosting rounds and leaves in trees
gbm_options = {
'num_boost_round': 100,
'num_leaves': ag.space.Int(lower=26, upper=66, default=36),
}
hyperparameters = { # hyperparameters of each model type
'GBM': gbm_options,
'NN': nn_options,
}
# Use at most 3 different hyperparameter configurations for each type of model
num_trials = 3
# Use Bayesian optimization routine with a local scheduler
search_strategy = 'auto'
hyperparameter_tune_kwargs = {
'num_trials': num_trials,
'scheduler' : 'local',
'searcher': search_strategy,
}
predictor_new_hpo = TabularPredictor(
label="count", problem_type="regression", eval_metric="rmse"
).fit(
train_data=train.drop(['casual', 'registered'], axis=1),
time_limit=600,
presets='best_quality', hyperparameters=hyperparameters, hyperparameter_tune_kwargs=hyperparameter_tune_kwargs)
No path specified. Models will be saved in: "AutogluonModels/ag-20230611_155623/"
Presets specified: ['best_quality']
Warning: hyperparameter tuning is currently experimental and may cause the process to hang.
Stack configuration (auto_stack=True): num_stack_levels=1, num_bag_folds=8, num_bag_sets=20
Beginning AutoGluon training ... Time limit = 600s
AutoGluon will save models to "AutogluonModels/ag-20230611_155623/"
AutoGluon Version: 0.6.2
Python Version: 3.7.10
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Thu May 4 09:55:30 UTC 2023
Train Data Rows: 10886
Train Data Columns: 13
Label Column: count
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 1990.23 MB
Train Data (Original) Memory Usage: 0.98 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Fitting DatetimeFeatureGenerator...
/usr/local/lib/python3.7/site-packages/autogluon/features/generators/datetime.py:59: FutureWarning: casting datetime64[ns, UTC] values to int64 with .astype(...) is deprecated and will raise in a future version. Use .view(...) instead.
good_rows = series[~series.isin(bad_rows)].astype(np.int64)
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('datetime', []) : 1 | ['datetime']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 7 | ['holiday', 'workingday', 'humidity', 'year', 'month', ...]
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
0.2s = Fit runtime
13 features in original data used to generate 17 features in processed data.
Train Data (Processed) Memory Usage: 1.1 MB (0.1% of available memory)
Data preprocessing and feature engineering runtime = 0.2s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
This metric's sign has been flipped to adhere to being higher_is_better. The metric score can be multiplied by -1 to get the metric value.
To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
WARNING: "NN" model has been deprecated in v0.4.0 and renamed to "NN_MXNET". Starting in v0.6.0, specifying "NN" or "NN_MXNET" will raise an exception. Consider instead specifying "NN_TORCH".
Fitting 2 L1 models ...
Hyperparameter tuning model: LightGBM_BAG_L1 ... Tuning model for up to 179.9s of the 599.8s of remaining time.
0%| | 0/3 [00:00<?, ?it/s] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
33%|███▎ | 1/3 [00:22<00:44, 22.24s/it] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
67%|██████▋ | 2/3 [00:45<00:22, 22.92s/it] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
100%|██████████| 3/3 [01:08<00:00, 22.95s/it]
Fitted model: LightGBM_BAG_L1/T1 ...
-40.2554 = Validation score (-root_mean_squared_error)
22.2s = Training runtime
0.0s = Validation runtime
Fitted model: LightGBM_BAG_L1/T2 ...
-38.8609 = Validation score (-root_mean_squared_error)
23.37s = Training runtime
0.0s = Validation runtime
Fitted model: LightGBM_BAG_L1/T3 ...
-38.5604 = Validation score (-root_mean_squared_error)
23.18s = Training runtime
0.0s = Validation runtime
Hyperparameter tuning model: NeuralNetMXNet_BAG_L1 ... Tuning model for up to 179.9s of the 530.81s of remaining time.
0%| | 0/3 [00:00<?, ?it/s] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
ray::_ray_fit() (pid=4939, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 49, in model_trial
time_limit=time_limit,
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 101, in fit_and_save_model
model.fit(**fit_args, time_limit=time_left)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 154, in _fit
return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 251, in _fit
n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 541, in _fit_folds
fold_fitting_strategy.after_all_folds_scheduled()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 536, in after_all_folds_scheduled
raise processed_exception
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 504, in after_all_folds_scheduled
time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/_private/worker.py", line 2280, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::_ray_fit() (pid=4939, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
33%|███▎ | 1/3 [00:06<00:12, 6.06s/it]2023-06-11 15:57:39,386 ERROR worker.py:400 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): The worker died unexpectedly while executing this task. Check python-core-worker-*.log files for more information.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
ray::_ray_fit() (pid=5000, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 49, in model_trial
time_limit=time_limit,
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 101, in fit_and_save_model
model.fit(**fit_args, time_limit=time_left)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 154, in _fit
return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 251, in _fit
n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 541, in _fit_folds
fold_fitting_strategy.after_all_folds_scheduled()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 536, in after_all_folds_scheduled
raise processed_exception
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 504, in after_all_folds_scheduled
time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/_private/worker.py", line 2280, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::_ray_fit() (pid=5000, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
67%|██████▋ | 2/3 [00:11<00:05, 5.70s/it]2023-06-11 15:57:44,832 ERROR worker.py:400 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): The worker died unexpectedly while executing this task. Check python-core-worker-*.log files for more information.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
ray::_ray_fit() (pid=5066, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 49, in model_trial
time_limit=time_limit,
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 101, in fit_and_save_model
model.fit(**fit_args, time_limit=time_left)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 154, in _fit
return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 251, in _fit
n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 541, in _fit_folds
fold_fitting_strategy.after_all_folds_scheduled()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 536, in after_all_folds_scheduled
raise processed_exception
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 504, in after_all_folds_scheduled
time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/_private/worker.py", line 2280, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::_ray_fit() (pid=5066, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
100%|██████████| 3/3 [00:16<00:00, 5.63s/it]
No model was trained during hyperparameter tuning NeuralNetMXNet_BAG_L1... Skipping this model.
Repeating k-fold bagging: 2/20
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 313.78s of the 513.8s of remaining time.
2023-06-11 15:57:50,187 ERROR worker.py:400 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): The worker died unexpectedly while executing this task. Check python-core-worker-*.log files for more information.
Fitting 8 child models (S2F1 - S2F8) | Fitting with ParallelLocalFoldFittingStrategy
-39.7819 = Validation score (-root_mean_squared_error)
41.55s = Training runtime
0.15s = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 290.65s of the 490.67s of remaining time.
Fitting 8 child models (S2F1 - S2F8) | Fitting with ParallelLocalFoldFittingStrategy
-38.487 = Validation score (-root_mean_squared_error)
41.9s = Training runtime
0.16s = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 268.0s of the 468.03s of remaining time.
Fitting 8 child models (S2F1 - S2F8) | Fitting with ParallelLocalFoldFittingStrategy
-38.1028 = Validation score (-root_mean_squared_error)
42.16s = Training runtime
0.16s = Validation runtime
Repeating k-fold bagging: 3/20
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 244.81s of the 444.83s of remaining time.
Fitting 8 child models (S3F1 - S3F8) | Fitting with ParallelLocalFoldFittingStrategy
-39.6161 = Validation score (-root_mean_squared_error)
60.32s = Training runtime
0.3s = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 221.23s of the 421.26s of remaining time.
Fitting 8 child models (S3F1 - S3F8) | Fitting with ParallelLocalFoldFittingStrategy
-38.3259 = Validation score (-root_mean_squared_error)
60.57s = Training runtime
0.31s = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 198.37s of the 398.39s of remaining time.
Fitting 8 child models (S3F1 - S3F8) | Fitting with ParallelLocalFoldFittingStrategy
-37.9204 = Validation score (-root_mean_squared_error)
65.43s = Training runtime
0.37s = Validation runtime
Repeating k-fold bagging: 4/20
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 170.92s of the 370.95s of remaining time.
Fitting 8 child models (S4F1 - S4F8) | Fitting with ParallelLocalFoldFittingStrategy
-39.5522 = Validation score (-root_mean_squared_error)
79.42s = Training runtime
0.44s = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 147.54s of the 347.57s of remaining time.
Fitting 8 child models (S4F1 - S4F8) | Fitting with ParallelLocalFoldFittingStrategy
-38.2599 = Validation score (-root_mean_squared_error)
78.83s = Training runtime
0.45s = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 124.96s of the 324.99s of remaining time.
Fitting 8 child models (S4F1 - S4F8) | Fitting with ParallelLocalFoldFittingStrategy
-37.8472 = Validation score (-root_mean_squared_error)
85.13s = Training runtime
0.54s = Validation runtime
Repeating k-fold bagging: 5/20
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 100.62s of the 300.64s of remaining time.
Fitting 8 child models (S5F1 - S5F8) | Fitting with ParallelLocalFoldFittingStrategy
-39.5237 = Validation score (-root_mean_squared_error)
98.04s = Training runtime
0.59s = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 77.78s of the 277.81s of remaining time.
Fitting 8 child models (S5F1 - S5F8) | Fitting with ParallelLocalFoldFittingStrategy
-38.2323 = Validation score (-root_mean_squared_error)
98.17s = Training runtime
0.64s = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 54.19s of the 254.21s of remaining time.
Fitting 8 child models (S5F1 - S5F8) | Fitting with ParallelLocalFoldFittingStrategy
-37.8113 = Validation score (-root_mean_squared_error)
104.41s = Training runtime
0.72s = Validation runtime
Completed 5/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 230.21s of remaining time.
-37.5326 = Validation score (-root_mean_squared_error)
0.24s = Training runtime
0.0s = Validation runtime
WARNING: "NN" model has been deprecated in v0.4.0 and renamed to "NN_MXNET". Starting in v0.6.0, specifying "NN" or "NN_MXNET" will raise an exception. Consider instead specifying "NN_TORCH".
Fitting 2 L2 models ...
Hyperparameter tuning model: LightGBM_BAG_L2 ... Tuning model for up to 103.46s of the 229.89s of remaining time.
0%| | 0/3 [00:00<?, ?it/s] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
33%|███▎ | 1/3 [00:23<00:46, 23.17s/it] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
67%|██████▋ | 2/3 [00:47<00:23, 23.66s/it] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
100%|██████████| 3/3 [01:11<00:00, 23.70s/it]
Fitted model: LightGBM_BAG_L2/T1 ...
-36.5025 = Validation score (-root_mean_squared_error)
23.13s = Training runtime
0.0s = Validation runtime
Fitted model: LightGBM_BAG_L2/T2 ...
-36.3887 = Validation score (-root_mean_squared_error)
23.95s = Training runtime
0.0s = Validation runtime
Fitted model: LightGBM_BAG_L2/T3 ...
-36.6849 = Validation score (-root_mean_squared_error)
23.9s = Training runtime
0.0s = Validation runtime
Hyperparameter tuning model: NeuralNetMXNet_BAG_L2 ... Tuning model for up to 103.46s of the 158.66s of remaining time.
0%| | 0/3 [00:00<?, ?it/s] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
ray::_ray_fit() (pid=8714, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 49, in model_trial
time_limit=time_limit,
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 101, in fit_and_save_model
model.fit(**fit_args, time_limit=time_left)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 154, in _fit
return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 251, in _fit
n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 541, in _fit_folds
fold_fitting_strategy.after_all_folds_scheduled()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 536, in after_all_folds_scheduled
raise processed_exception
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 504, in after_all_folds_scheduled
time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/_private/worker.py", line 2280, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::_ray_fit() (pid=8714, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
33%|███▎ | 1/3 [00:05<00:11, 5.89s/it] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
2023-06-11 16:03:51,868 ERROR worker.py:400 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): The worker died unexpectedly while executing this task. Check python-core-worker-*.log files for more information.
ray::_ray_fit() (pid=8775, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 49, in model_trial
time_limit=time_limit,
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 101, in fit_and_save_model
model.fit(**fit_args, time_limit=time_left)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 154, in _fit
return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 251, in _fit
n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 541, in _fit_folds
fold_fitting_strategy.after_all_folds_scheduled()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 536, in after_all_folds_scheduled
raise processed_exception
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 504, in after_all_folds_scheduled
time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/_private/worker.py", line 2280, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::_ray_fit() (pid=8775, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
67%|██████▋ | 2/3 [00:11<00:05, 5.97s/it] Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
2023-06-11 16:03:57,764 ERROR worker.py:400 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): The worker died unexpectedly while executing this task. Check python-core-worker-*.log files for more information.
ray::_ray_fit() (pid=8868, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 49, in model_trial
time_limit=time_limit,
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/model_trial.py", line 101, in fit_and_save_model
model.fit(**fit_args, time_limit=time_left)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 154, in _fit
return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 251, in _fit
n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 541, in _fit_folds
fold_fitting_strategy.after_all_folds_scheduled()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 536, in after_all_folds_scheduled
raise processed_exception
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 504, in after_all_folds_scheduled
time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/_private/worker.py", line 2280, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::_ray_fit() (pid=8868, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 375, in _ray_fit
time_limit=time_limit_fold, **resources, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 703, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/mxnet/tabular_nn_mxnet.py", line 135, in _fit
try_import_mxnet()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/utils/try_import.py", line 40, in try_import_mxnet
import mxnet as mx
File "/usr/local/lib/python3.7/site-packages/mxnet/__init__.py", line 33, in <module>
from . import contrib
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/__init__.py", line 30, in <module>
from . import text
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/__init__.py", line 23, in <module>
from . import embedding
File "/usr/local/lib/python3.7/site-packages/mxnet/contrib/text/embedding.py", line 37, in <module>
from ... import numpy_extension as _mx_npx
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/__init__.py", line 23, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/numpy_extension/image.py", line 20, in <module>
from ..image import * # pylint: disable=wildcard-import, unused-wildcard-import
File "/usr/local/lib/python3.7/site-packages/mxnet/image/__init__.py", line 22, in <module>
from . import image
File "/usr/local/lib/python3.7/site-packages/mxnet/image/image.py", line 38, in <module>
import cv2
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.7/site-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.7/site-packages/cv2/gapi/__init__.py", line 301, in <module>
cv.gapi.wip.GStreamerPipeline = cv.gapi_wip_gst_GStreamerPipeline
AttributeError: module 'cv2' has no attribute 'gapi_wip_gst_GStreamerPipeline'
100%|██████████| 3/3 [00:18<00:00, 6.11s/it]
No model was trained during hyperparameter tuning NeuralNetMXNet_BAG_L2... Skipping this model.
Repeating k-fold bagging: 2/20
Fitting model: LightGBM_BAG_L2/T1 ... Training model for up to 140.23s of the 140.21s of remaining time.
2023-06-11 16:04:03,782 ERROR worker.py:400 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): The worker died unexpectedly while executing this task. Check python-core-worker-*.log files for more information.
Fitting 8 child models (S2F1 - S2F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.3886 = Validation score (-root_mean_squared_error)
42.95s = Training runtime
0.14s = Validation runtime
Fitting model: LightGBM_BAG_L2/T2 ... Training model for up to 116.51s of the 116.49s of remaining time.
Fitting 8 child models (S2F1 - S2F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.2505 = Validation score (-root_mean_squared_error)
43.16s = Training runtime
0.14s = Validation runtime
Fitting model: LightGBM_BAG_L2/T3 ... Training model for up to 93.09s of the 93.07s of remaining time.
Fitting 8 child models (S2F1 - S2F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.4611 = Validation score (-root_mean_squared_error)
44.04s = Training runtime
0.24s = Validation runtime
Completed 2/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the 68.65s of remaining time.
-36.1696 = Validation score (-root_mean_squared_error)
0.25s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 531.79s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20230611_155623/")
predictor_new_hpo.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
model score_val pred_time_val fit_time pred_time_val_marginal fit_time_marginal stack_level can_infer fit_order
0 WeightedEnsemble_L3 -36.169577 2.328148 388.058489 0.000806 0.246125 3 True 8
1 LightGBM_BAG_L2/T2 -36.250479 2.085000 343.776040 0.140926 43.157349 2 True 6
2 LightGBM_BAG_L2/T1 -36.388639 2.084663 343.570397 0.140589 42.951706 2 True 5
3 LightGBM_BAG_L2/T3 -36.461067 2.186416 344.655014 0.242342 44.036323 2 True 7
4 WeightedEnsemble_L2 -37.532609 1.358947 202.816522 0.000916 0.241940 2 True 4
5 LightGBM_BAG_L1/T3 -37.811320 0.722558 104.408426 0.722558 104.408426 1 True 3
6 LightGBM_BAG_L1/T2 -38.232258 0.635473 98.166155 0.635473 98.166155 1 True 2
7 LightGBM_BAG_L1/T1 -39.523687 0.586043 98.044110 0.586043 98.044110 1 True 1
Number of models trained: 8
Types of models trained:
{'WeightedEnsembleModel', 'StackerEnsembleModel_LGB'}
Bagging used: True (with 8 folds)
Multi-layer stack-ensembling used: True (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
Plot summary of models saved to file: AutogluonModels/ag-20230611_155623/SummaryOfModels.html
*** End of fit() summary ***
{'model_types': {'LightGBM_BAG_L1/T1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T3': 'StackerEnsembleModel_LGB',
'WeightedEnsemble_L2': 'WeightedEnsembleModel',
'LightGBM_BAG_L2/T1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T3': 'StackerEnsembleModel_LGB',
'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
'model_performance': {'LightGBM_BAG_L1/T1': -39.52368745300766,
'LightGBM_BAG_L1/T2': -38.232258025837844,
'LightGBM_BAG_L1/T3': -37.81131960847469,
'WeightedEnsemble_L2': -37.53260915677494,
'LightGBM_BAG_L2/T1': -36.38863903351911,
'LightGBM_BAG_L2/T2': -36.25047854854333,
'LightGBM_BAG_L2/T3': -36.46106727870545,
'WeightedEnsemble_L3': -36.16957696672271},
'model_best': 'WeightedEnsemble_L3',
'model_paths': {'LightGBM_BAG_L1/T1': '/root/nd009t-c1-intro-to-ml-templates/cd0385-project-starter/project/AutogluonModels/ag-20230611_155623/models/LightGBM_BAG_L1/T1/',
'LightGBM_BAG_L1/T2': '/root/nd009t-c1-intro-to-ml-templates/cd0385-project-starter/project/AutogluonModels/ag-20230611_155623/models/LightGBM_BAG_L1/T2/',
'LightGBM_BAG_L1/T3': '/root/nd009t-c1-intro-to-ml-templates/cd0385-project-starter/project/AutogluonModels/ag-20230611_155623/models/LightGBM_BAG_L1/T3/',
'WeightedEnsemble_L2': 'AutogluonModels/ag-20230611_155623/models/WeightedEnsemble_L2/',
'LightGBM_BAG_L2/T1': '/root/nd009t-c1-intro-to-ml-templates/cd0385-project-starter/project/AutogluonModels/ag-20230611_155623/models/LightGBM_BAG_L2/T1/',
'LightGBM_BAG_L2/T2': '/root/nd009t-c1-intro-to-ml-templates/cd0385-project-starter/project/AutogluonModels/ag-20230611_155623/models/LightGBM_BAG_L2/T2/',
'LightGBM_BAG_L2/T3': '/root/nd009t-c1-intro-to-ml-templates/cd0385-project-starter/project/AutogluonModels/ag-20230611_155623/models/LightGBM_BAG_L2/T3/',
'WeightedEnsemble_L3': 'AutogluonModels/ag-20230611_155623/models/WeightedEnsemble_L3/'},
'model_fit_times': {'LightGBM_BAG_L1/T1': 98.04410982131958,
'LightGBM_BAG_L1/T2': 98.16615533828735,
'LightGBM_BAG_L1/T3': 104.40842580795288,
'WeightedEnsemble_L2': 0.24194049835205078,
'LightGBM_BAG_L2/T1': 42.951706409454346,
'LightGBM_BAG_L2/T2': 43.15734934806824,
'LightGBM_BAG_L2/T3': 44.0363233089447,
'WeightedEnsemble_L3': 0.2461252212524414},
'model_pred_times': {'LightGBM_BAG_L1/T1': 0.586043119430542,
'LightGBM_BAG_L1/T2': 0.6354734897613525,
'LightGBM_BAG_L1/T3': 0.722557544708252,
'WeightedEnsemble_L2': 0.00091552734375,
'LightGBM_BAG_L2/T1': 0.14058923721313477,
'LightGBM_BAG_L2/T2': 0.14092612266540527,
'LightGBM_BAG_L2/T3': 0.2423417568206787,
'WeightedEnsemble_L3': 0.0008058547973632812},
'num_bag_folds': 8,
'max_stack_level': 3,
'model_hyperparams': {'LightGBM_BAG_L1/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L2': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L3': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True}},
'leaderboard': model score_val pred_time_val fit_time \
0 WeightedEnsemble_L3 -36.169577 2.328148 388.058489
1 LightGBM_BAG_L2/T2 -36.250479 2.085000 343.776040
2 LightGBM_BAG_L2/T1 -36.388639 2.084663 343.570397
3 LightGBM_BAG_L2/T3 -36.461067 2.186416 344.655014
4 WeightedEnsemble_L2 -37.532609 1.358947 202.816522
5 LightGBM_BAG_L1/T3 -37.811320 0.722558 104.408426
6 LightGBM_BAG_L1/T2 -38.232258 0.635473 98.166155
7 LightGBM_BAG_L1/T1 -39.523687 0.586043 98.044110
pred_time_val_marginal fit_time_marginal stack_level can_infer \
0 0.000806 0.246125 3 True
1 0.140926 43.157349 2 True
2 0.140589 42.951706 2 True
3 0.242342 44.036323 2 True
4 0.000916 0.241940 2 True
5 0.722558 104.408426 1 True
6 0.635473 98.166155 1 True
7 0.586043 98.044110 1 True
fit_order
0 8
1 6
2 5
3 7
4 4
5 3
6 2
7 1 }
# Use the new predictor to make a prediction
prediction_new_hpo = predictor_new_hpo.predict(test)
prediction_new_hpo = {'datetime': test['datetime'], 'Pred_count': prediction_new_hpo}
prediction_new_hpo = pd.DataFrame(data=prediction_new_hpo)
prediction_new_hpo.head()
| datetime | Pred_count | |
|---|---|---|
| 0 | 2011-01-20 00:00:00 | 10.239550 |
| 1 | 2011-01-20 01:00:00 | 6.499575 |
| 2 | 2011-01-20 02:00:00 | 6.490079 |
| 3 | 2011-01-20 03:00:00 | 6.275881 |
| 4 | 2011-01-20 04:00:00 | 6.275962 |
# Remember to set all negative values to zero
prediction_new_hpo[prediction_new_hpo['Pred_count']<0] = 0
# Same submitting predictions
submission_new_hpo = pd.read_csv('submission.csv')
submission_new_hpo["count"] = prediction_new_hpo['Pred_count']
submission_new_hpo.to_csv("submission_new_hpo.csv", index=False)
!kaggle competitions submit -c bike-sharing-demand -f submission_new_hpo.csv -m "new features with hyperparameters"
100%|█████████████████████████████████████████| 188k/188k [00:00<00:00, 380kB/s] Successfully submitted to Bike Sharing Demand
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName date description status publicScore privateScore --------------------------- ------------------- --------------------------------- -------- ----------- ------------ submission_new_hpo.csv 2023-06-11 16:06:29 new features with hyperparameters complete 0.48752 0.48752 submission_new_features.csv 2023-06-11 15:56:21 new features complete 0.68994 0.68994 submission.csv 2023-06-11 15:29:46 first raw submission complete 1.79005 1.79005
0.48752¶# Taking the top model score from each training run and creating a line plot to show improvement
# You can create these in the notebook and save them to PNG or use some other tool (e.g. google sheets, excel)
fig = pd.DataFrame(
{
"model": ["initial", "add_features", "hpo"],
"score": [53.092490, 30.084351, 36.169577]
}
).plot(x="model", y="score", figsize=(8, 6)).get_figure()
fig.savefig('model_train_score.png')
# Take the 3 kaggle scores and creating a line plot to show improvement
fig = pd.DataFrame(
{
"test_eval": ["initial", "add_features", "hpo"],
"score": [1.79005, 0.68994, 0.48752]
}
).plot(x="test_eval", y="score", figsize=(8, 6)).get_figure()
fig.savefig('model_test_score.png')
# The 3 hyperparameters we tuned with the kaggle score as the result
hyperparams_df = pd.DataFrame({
"model": ["initial_model", "add_features_model", "hpo_model"],
"hpo1": ['default_vals', 'default_vals', 'GBM: num_leaves: lower=26, upper=66'],
"hpo2": ['default_vals', 'default_vals', 'NN: dropout_prob: 0.0, 0.5'],
"hpo3": ['default_vals', 'default_vals', 'GBM: num_boost_round: 100'],
"score": [1.79005, 0.68994, 0.48752]
})
hyperparams_df.head()
| model | hpo1 | hpo2 | hpo3 | score | |
|---|---|---|---|---|---|
| 0 | initial_model | default_vals | default_vals | default_vals | 1.79005 |
| 1 | add_features_model | default_vals | default_vals | default_vals | 0.68994 |
| 2 | hpo_model | GBM: num_leaves: lower=26, upper=66 | NN: dropout_prob: 0.0, 0.5 | GBM: num_boost_round: 100 | 0.48752 |
def plot_series(time, series, format="-", start=0, end=None, label=None):
plt.plot(time[start:end], series[start:end], format, label=label)
plt.xlabel("Time")
plt.ylabel("Value")
if label:
plt.legend(fontsize=14)
plt.grid(True)
sub_new_features = pd.read_csv('submission_new_features.csv')
# Plotting time series of testing data
import matplotlib.pyplot as plt
series = train["count"].to_numpy()
time = train["datetime"].to_numpy()
plt.figure(figsize=(350, 15))
plot_series(time, series)
plt.title("Training data time series graph")
plt.show()
# Plotting time series of testing data
sub_new_features.loc[:, "datetime"] = pd.to_datetime(sub_new_features.loc[:, "datetime"])
series1 = sub_new_features["count"].to_numpy()
time1 = sub_new_features["datetime"].to_numpy()
plt.figure(figsize=(350, 15))
plot_series(time1, series1)
plt.title("Testing data time series graph")
plt.show()
